Administration Guide

|The LAPI shared memory execution model

|It is desirable for communication protocol clients to use an efficient |shared memory protocol rather than the switch adapter or double copy (with |communicating by a share segment) when the tasks are on the same node. |This reduces switch congestion and increases performance. The LAPI |shared memory model is designed for this purpose. Figure 43 and Figure 44 show the different approaches of executing |LAPI_Put when task 0 and task 1 are on the same node. The user |interface is identical in the shared and nonshared memory LAPI communication |paths. |

|Figure 43. LAPI_Put without shared memory

|View figure.
|

|Figure 44. LAPI_Put with shared memory

|View figure.

|Cross memory kernel extension

|A kernel extension is provided to support the LAPI shared memory model |which is loaded as part of AIX initialization. This kernel extension |allows one task to export a portion of its address space to another task of |the parallel application. The exported portion of the address space is |attached to a companion the address space of the companion task and the data |transfer is done by a simple copy. A separate shared memory region is |required for exchanging meta data describing the exported regions and allowing |for handshaking between the communicating tasks of the parallel |application. This shared memory region can also be used for |transferring small or noncontiguous messages.

|As part of CSS installation, the configuration method for the kernel |extension is added to the Config_Rules ODM database to be run in phase 3 of |the system initialization. This ensures that the kernel extension is |available for any CSS installation, regardless of the switch adapter or lack |of switch adapter.

|Functional flow in shared memory in LAPI

|Each task which communicates with other tasks with shared memory has a |message queue. Shared memory task X processes commands off of the head |of the task X message queue and tasks other than task X update the tail of the |task X message queue. There is also a data area associated with each |command structure. Each data transfer with shared memory can be done in |two possible ways: |

• |For small messages, the message is copied into a shared memory location by |the message origin. It is subsequently copied out of the shared memory |by the message target.
• |For large messages, the segment identifiers for the segments containing |the message are stored in a kernel buffer with a new kernel extension at the |message origin. On the message target, the segment identifers are |attached and the transfer is completed by a simple copy. |

|LAPI shared memory requirements and limitations

|Following are the requirements and limitations for using LAPI shared |memory: |

• |The environment variable LAPI_USE_SHM must be set to either YES or |ONLY. It is case insensitive. If ONLY is set, LAPI only uses the |shared memory mechanism and an error message is returned if all tasks are not |on the same node or shared memory setup is not successful. If |LAPI_USE_SHM is set to YES, the shared memory path is used on tasks within a |node unless the shared memory initialization fails. In that case, the |switch path is used. For those tasks on different nodes, the switch |path is used.
• |The kernel extension must be loaded.
• |Shared memory LAPI must be used on AIX 4.3 or later.
• |The maximum number of tasks supported on a node is 32.
• |The shared memory transport mechanism can be used only in the POE |environment. |