The network subsystem uses a memory management facility that revolves around a data structure called an mbuf. Mbufs are mostly used to store data for incoming and outbound network traffic. Having mbuf pools of the right size can have a very positive effect on network performance. If the mbuf pools are configured incorrectly, both network and system performance can suffer. In AIX Version 4, the mbuf parameters are automatically tuned based on the system configuration, particularly the number of communications adapters, and network requirements.
The mbuf management facility controls different buffer sizes that can range from 32 bytes up to 16384 bytes. The pools are created from system memory by making an allocation request to the Virtual Memory Manager (VMM). The pools consist of pinned pieces of kernel virtual memory in which they always reside in physical memory and are never paged out. The result is that the real memory available for paging in application programs and data has been decreased by the amount that the mbuf pools have been increased.
The network memory pool is split evenly among each processor. Each sub-pool is then split up into buckets of 32-16384 bytes. Each bucket can borrow memory from other buckets on the same processor but a processor cannot borrow memory from another processor's network memory pool. When a network service needs to transport data, it can call a kernel service such as m_get() to obtain a memory buffer. If the buffer is already available and pinned, it can get it immediately. If the upper limit has not been reached and the buffer is not pinned, then a buffer is allocated and pinned. Once pinned, the memory stays pinned but can be freed back to the network pool. If the number of free buffers reaches a high-water mark, then a certain number is unpinned and given back to the system for general use. This unpinning is done by the netm() kernel process. The caller of the m_get() subroutine can specify whether to wait for a network memory buffer. If M_DONTWAIT is specified and no pinned buffers are available at that time, a failed counter is incremented. If M_WAIT is specified, the process is put to sleep until the buffer can be allocated and pinned.
Use the command netstat -m to detect shortages of network memory (mbufs/clusters).
An upper limit is placed on how much of RAM can be used for network memory. You can tune this parameter by setting maxmbuf or thewall. The unit of thewall is in 1 KB, so that 16384 indicates 16 MB of RAM.
The default value for thewall in operating system version
The maximum value of thewall is 64 MB prior to operating system version 4.3.0, 128 MB in operating system version AIX 4.3.0 and version AIX 4.3.1, and 1 GB in AIX 4.3.2 and higher (systems that are not Common Hardware Reference Platform (CHRP) are limited to 256 MB).
The value of maxmbuf is also used to limit how much real memory can be used by the communications subsystem. You can view this by running the command lsattr -E -l sys0. If maxmbuf is greater than 0 (can be changed using the chdev or smitty commands), the maxmbuf value is used regardless of the value of thewall. Previously, the upper limit on mbufs was the higher value of maxmbuf or thewall.
maxmbuf is set to 0. The value of 0 indicates that the system default (thewall) should be used.
The limit on network memory (thewall or maxmbuf) is also used to limit how much memory can be used for streams. The tunable parameter called strthresh (default value of 85 percent of thewall/maxmbuf) specifies that once the total amount of allocated memory has reached 85 percent, no more memory can be given to STREAMS.
Similarly, another threshold called sockthresh (also defaults to 85 percent) specifies that once the total amount of allocated memory has reached 85 percent of thewall/maxmbuf, no new socket connections can occur (the socket() and socketpair() system calls return with ENOBUFS). Use the no command to tune these thresholds.