IBM POWER-based SMP hardware systems contain several multichip modules (MCMs), each containing multiple processors. System memory is attached to these MCMs. While any processor can access all of the memory in the system, a processor has faster access, and higher bandwidth, when addressing memory that is attached to its own MCM rather than memory attached to the other MCMs in the system.
AIX has optional support to recognize the division of system memory among the MCMs. If the memory affinity support is enabled, AIX attempts to satisfy a page fault using memory attached to the MCM containing the processor that caused the page fault. This may provide a performance benefit to the application.
Using the memory affinity support on AIX is a two step process. The following vmtune command will enable the support:
vmtune -y 0|1 (0 disabled, 1 enabled)
This action will only tell AIX to organize its data structures along MCM boundaries. The default memory allocation policy rotates among the MCMs. In order to obtain preferential local MCM memory allocation, an application must export the MEMORY_AFFINITY environment variable as follows:
MEMORY_AFFINITY=MCM
This behavior is propogated across a fork. However, for this behavior to be retained across an exec, the variable must be contained in the environment string passed to the exec function call.
The effect local MCM memory allocation will have on a specific application is difficult to predict. Some applications are unaffected, some might improve, and others might degrade.
Most applications must be bound to processors to get a performance benefit from memory affinity. This is needed to prevent the AIX dispatcher from moving the application to processors in different MCMs while the application executes.
The most likely way to obtain a benefit from memory affinity is to limit the application to running only on the processors contained in a single MCM. This can be done with the bindprocessor command and the bindprocessor() function. It can also be done with the resource set affinity commands and services.
When the application requires more processors than contained in a single MCM, the performance benefit through memory affinity depends on the memory allocation and access patterns of the various threads in the application. Applications with threads that individually allocate and reference unique data areas may see improved performance. Applications that share memory among all the threads are more likely to get a degradation from memory affinity.
The vmtune command and Tuning VMM Page Replacement with the vmtune Command.
The bindprocessor command or sub-routine.
WLM Class Attributes and Resource Set Attributes.