University of Wisconsin-Madison  My UW
Computing @ UW-Madison

Installing AIX
Tuning AIX
Patching, Fixes, APARs
Compilers
Tips & Tricks

TSM Clients

IBM pSeries Support
TechLib
IBM Redbooks
IBM Documentation Library

Back to DoIT

Notes and Observations While Tuning AIX:

  1. Oracle, as well as other database systems, has the ability to use asynchronous I/O. To improve performance you can add AIO server processes based on the number of disks (real or emulated) that will be accessed. There are two settings: AIO_MAX and AIO_MIN. AIO_MAX is the maximum number of asynchronous I/O processes that will ever be running. AIO_MIN, conversely, is the minimum number, and what is started by default upon boot.

    Recommendations range from 5 to 10 AIO processes per disk. Generally we keep the minimum tuned to half of the maximum value, and the maximum set to 5 times the number of drives. If you watch to see if the maximum is reached you may opt to increase this again to 10 times the number. You can watch AIO processes using the command "pstat -a | grep aios | wc -l". Also watch the I/O wait time, as a shortage of AIO servers will manifest itself in longer wait times.

    A reboot is necessary to enable this change.

  2. Adjust AIO max requests in conjunction with #1. It defaults to 4096. Different drives have different queue depths which will affect this setting. Enterprise Storage System (IBM 2105) drives have a queue depth of 64, while SSA drives have a queue depth of 3. If you divide the current max request setting by the number of drives then you should get a good idea if it needs to be raised or not (the result of your math should be greater than the queue depth).

    A reboot is necessary to enable this change.

  3. Use lockstat to track kernel locks. The kernel needs to be built with lock instrumentation enabled. The command "bosboot -L" will do that. FINISH THIS

  4. Use /usr/samples/kernel/emstat to view emulated instructions. Because AIX runs on multiple architectures, to maintain compatibility between them AIX emulates CPU instructions that are not present in the current system architecture. This results in slower performance. When possible, always compile software to use the specific architecture you're on or the common instruction set supported by all platforms. The native IBM VisualAge compilers compile to the common instruction set by default; gcc compiles specifically for the native platform. See the guide on tuning Apache for more information about compiler-based optimizations.

  5. Increase the tcp_sendspace and tcp_recvspace. With RFC 1323 support turned on these spaces can grow to 4 GB, otherwise they are limited to 64 KB. These settings permit more outstanding packets between machines, thereby allowing more packets to be in transit at any given time. On fast networks this is a good thing. You should keep these values synchronized between machines that often communicate with each other. The 'no' command sets these, and to adjust them several parameters must be set. The default at boot is 16 KB, so setting these shortly after boot is recommended.

    Selective acknowledgement (RFC 2018) allows TCP to recover from multiple losses within the same window, and gives better performance on lossy networks and internetworks.

    The sb_max parameter should always be twice what the tcp_sendspace and tcp_recvspace are set to. Bad things will happen if it isn't.

    All of these settings should be multiples of 4096, too.

    You can also adjust the same parameters for UDP, too.

    # RFC1323 support (required to change tcp_sendspace)
    /usr/sbin/no -o rfc1323=1

    # RFC2018 support (selective acknowledgement - required with RFC1323)
    /usr/sbin/no -o sack=1

    # Increase the upper bound on TCP & UDP socket buffers
    /usr/sbin/no -o sb_max=262144

    # TCP send & receive spaces
    /usr/sbin/no -o tcp_sendspace=131072
    /usr/sbin/no -o tcp_recvspace=131072

    # UDP send & receive spaces
    /usr/sbin/no -o udp_sendspace=131072
    /usr/sbin/no -o udp_recvspace=131072

  6. Disable some various forms of Denial of Service attacks. While this isn't going to net a performance boost, it will keep people from sapping your performance and availability later on:

    # Socket queue defense against SYN attacks
    /usr/sbin/no -o clean_partial_conns=1

    # Disable the sending and receiving of redirects
    /usr/sbin/no -o ipignoreredirects=1
    /usr/sbin/no -o ipsendredirects=0

    # Disable the acceptance of source routed packets
    /usr/sbin/no -o ipsrcroutesend=0

    # Do not respond to directed broadcasts
    /usr/sbin/no -o directed_broadcast=0

    # Prevent address mask queries
    /usr/sbin/no -o icmpaddressmask=0

  7. Decrease the interpacket gap. This parameter is intended to prevent a machine from hogging a subnet when it generates a lot of traffic. However, on modern switched networks the machine is usually never contending with others (the nature of switching). Decreasing this setting increases the rate at which packets can flow. The effect of this change is noticable when the average packet size is low.

  8. Equal size paging sets, one per volume. Of course, the best way to optimize paging is to have enough RAM so you never page. However, spreading your paging spaces equally across multiple volumes is the next best thing.

  9. Specify Ethernet NIC line speeds. Autonegotiation for NICs often fails to find the optimal speed. It often fails, period, leaving the switch set to 10 Mbps half-duplex and the server at 100 Mbps full-duplex. The results of that are modem speeds on your NIC. If you set the NIC to the proper line speeds then you won't have to worry.

    This setting is held in the ODM, so if you delete and re-add the NIC for any reason you will need to specify it again.

  10. entstat -d en0 FINISH

  11. mbufs / netstat -m / thewall FINISH

  12. discussion of vmtune FINISH

 

Copyright © 2003 The Board of Regents of the University of Wisconsin System