Performance Management Guide

Detecting Instruction Emulation with the emstat Tool

The POWER-based architecture deleted (that is, no longer supports) 35 POWER instructions (5 for the PowerPC 601 RISC Microprocessor "bridge"). To maintain compatibility with older binaries (which may contain these deleted instructions) the AIX Version 4 kernel includes emulation routines that provide support for the deleted instructions. Attempting to execute a deleted instruction results in an illegal instruction exception. The kernel decodes the illegal instruction, and if it is a deleted instruction, the kernel runs an emulation routine that functionally emulates the instruction.

However, depending upon the execution frequency of deleted instructions and the deleted instruction emulation path lengths, emulation can result in varying degrees of performance degradation due to kernel context switch and instruction emulation overhead. Even a very small percentage of emulation might result in a big performance difference. The following table shows estimated instruction path lengths for several of the deleted instructions:

Most emulation problems are usually seen on PowerPC 604 RISC Microprocessor systems. A typical example is a PowerPC 601 RISC Microprocessor system that gets upgraded to a PowerPC 604 RISC Microprocessor system. If performance slows down instead of speeding up, it is most likely due to emulation.

The solution to emulation is to recompile the application in common mode. The default architecture platform for compilations on AIX Version 4 is common architecture. However, the default architecture on AIX Version 3 was for POWER family, POWER2, and PowerPC 601 RISC Microprocessor. If these binaries ran on a PowerPC 604 RISC Microprocessor, some instructions could get emulated.

If binaries need to be compiled on AIX Version 3, then the Full Common Mode APAR must be installed on that AIX Version 3 system, and the application must be compiled with the following options:

-qarch=com -qdebug=useabs -bI:/lib/FCM/lowsys.exp

Instructions that are not common on all platforms must be removed from code written in assembler, because recompilation is only effective for high-level source code. Routines coded in assembler must be changed so that they do not use missing instructions, because recompilation has no effect in this case.

The first step is to determine if instruction emulation is occurring by using the emstat tool.

To determine whether the emstat program is installed and available, run the following command:

# lslpp -lI bos.adt.samples

This tool resides in the /usr/samples/kernel directory.

With operating system versions later than AIX 4.3.3, this command ships with the bos.perf.tools fileset and has a slightly modified output. To determine whether the emstat program is installed and available, run the following command:

# lslpp -lI bos.perf.tools

The emstat command works similarly to the vmstat command in that you specify an interval time in seconds, and optionally, the number of intervals. The value in the first column is the cumulative count since system boot, while the value in the second column is the number of instructions emulated during that interval. Emulations on the order of many thousands per second can have an impact on performance.

The following is an example of output from issuing the emstat 1 command in AIX 4.3.3:

# /usr/samples/kernel/emstat 1

emstat total count      emstat interval count
       21951937                21951937
       21967488                   15551
       21974877                    7389
       21994354                   19477
       22007329                   12975
       22018171                   10842
       22018171                       0

The following is an example of output from issuing the emstat 1 command in AIX 5.1 or later:

# emstat 1

Emulation   Emulation
SinceBoot       Delta
        0           0
        0           0
        0           0

Once emulation has been detected, the next step is to determine which application is emulating instructions. This is much harder to determine. One way is to run only one application at a time and monitor it with the emstat program. Sometimes certain emulations cause a trace hook to be encountered. This can be viewed in the ASCII trace report file with the words PROGRAM CHECK. The process/thread associated with this trace event is emulating instructions either due to its own code emulating instructions, or it is executing library or code in other modules that are emulating instructions.

Detecting Alignment Exceptions with the alstat Tool

AIX compilers perform natural alignment of data types. For example, data of type short, which is 2 bytes long, is padded automatically to 4 bytes by the compiler. Common programming practices such as typecasting and usage of alignment pragmas can cause application data to be aligned incorrectly. POWER-based optimization assumes correct alignment of data. Thus, fetching misaligned data may require multiple memory accesses where a single access should have sufficed. Misalignment of data can cause the hardware to generate an alignment exception, which would force the kernel to simulate the needed memory accesses. As with the case of instrution emulation, this can degrade application performance.

Starting with AIX 5.1, the alstat tool packaged with bos.perf.tools can be used to detect if alignment exceptions are occurring. To show alignment exceptions on a per-CPU basis, use the -v option.

Because alstat and emstat are the same binary, either of these tools can be used to sho instruction emulation and alignment exceptions. To show instruction emulation, use the -e option on alstat. To show alignment exceptions, use the -a option on emstat.

The output for alstat looks similar to the following:

# alstat -e 1
  Alignment  Alignment  Emulation  Emulation
  SinceBoot      Delta  SinceBoot      Delta
          0          0          0          0
          0          0          0          0
          0          0          0          0