Commands Reference, Volume 5

sar Command

Purpose

Collects, reports, or saves system activity information.

/usr/sbin/sar [ { -A | [ -a ] [ -b ] [ -c ] [ -d ][ -k ] [ -m ] [ -q ] [ -r ] [ -u ] [ -V ] [ -v ] [ -w ] [ -y ] } ] [ -P ProcessorIdentifier, ... | ALL ] [ -ehh [ :mm [ :ss ] ] ] [ -fFile ] [ -iSeconds ] [ -oFile ] [ -shh [ :mm [ :ss ] ] ] [ Interval [ Number ] ]

Description

The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the Number and Interval parameters, writes information the specified number of times spaced at the specified intervals in seconds. The default sampling interval for the Number parameter is 1 second. The collected data can also be saved in the file specified by the -o File flag.

The sar command extracts and writes to standard output records previously saved in a file. This file can be either the one specified by the -f flag or, by default, the standard system activity daily data file, the /var/adm/sa/sadd file, where the dd parameter indicates the current day.

Without the -P flag, the sar command reports system-wide (global among all processors) statistics, which are calculated as averages for values expressed as percentages, and as sums otherwise. If the -P flag is given, the sar command reports activity which relates to the specified processor or processors. If -P ALL is given, the sar command reports statistics for each individual processor, followed by system-wide statistics.

You can select information about specific system activities using flags. Not specifying any flags selects only system unit activity. Specifying the -A flag selects all activities.

The default version of the sar command (CPU utilization report) might be one of the first facilities the user runs to begin system activity investigation, because it monitors major system resources. If CPU utilization is near 100 percent (user + system), the workload sampled is CPU-bound. If a considerable percentage of time is spent in I/O wait, it implies that CPU execution is blocked waiting for disk I/O. The I/O may be required file accesses or it may be I/O associated with paging due to a lack of sufficient memory.

Note: The time the system spends waiting for remote file access is not accumulated in the I/O wait time. If CPU utilization and I/O wait time for a task are relatively low, and the response time is not satisfactory, consider investigating how much time is being spent waiting for remote I/O. Since no high-level command provides statistics on remote I/O wait, trace data may be useful in observing this.

Methods Used to Compute CPU Disk I/O Wait Time

AIX 4.3.3 and later contains enhancements to the method used to compute the percentage of CPU time spent waiting on disk I/O (wio time).The method used in AIX 4.3.2 and earlier versions of the operating system can, under certain circumstances, give an inflated view of wio time on SMPs. The wio time is reported by the commands sar (%wio), vmstat (wa) and iostat (% iowait).

The method used in AIX 4.3.2 and earlier versions is as follows: At each clock interrupt on each processor (100 times a second per processor), a determination is made as to which of the four categories (usr/sys/wio/idle) to place the last 10 ms of time. If the CPU was busy in usr mode at the time of the clock interrupt, then usr gets the clock tick added into its category. If the CPU was busy in kernel mode at the time of the clock interrupt, then the sys category gets the tick. If the CPU was not busy, a check is made to see if any I/O to disk is in progress. If any disk I/O is in progress, the wio category is incremented. If no disk I/O is in progress and the CPU is not busy, the idle category gets the tick. The inflated view of wio time results from all idle CPUs being categorized as wio regardless of the number of threads waiting on I/O. For example, systems with just one thread doing I/O could report over 90 percent wio time regardless of the number of CPUs it has.

The method used in operating system AIX 4.3.3 and later is as follows: The change in operating system AIX 4.3.3 is to only mark an idle CPU as wio if an outstanding I/O was started on that CPU. This method can report much lower wio times when just a few threads are doing I/O and the system is otherwise idle. For example, a system with four CPUs and one thread doing I/O will report a maximum of 25 percent wio time. A system with 12 CPUs and one thread doing I/O will report a maximum of 8 percent wio time. NFS client reads/writes go through the VMM, and the time that biods spend in the VMM waiting for an I/O to complete is now reported as I/O wait time.

If multiple samples and multiple reports are desired, it is convenient to specify an output file for the sar command. Direct the standard output data from the sar command to /dev/null and run the sar command as a background process. The syntax for this is:

sar -A -o data.file interval count > /dev/null &

All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed with the sar command using the -f option.

The sar command calls a process named sadc to access system data. Two shell scripts (/usr/lib/sa/sa1 and /usr/lib/sa/sa2) are structured to be run by the cron command and provide daily statistics and reports. Sample stanzas are included (but commented out) in the /var/spool/cron/crontabs/adm crontab file to specify when the cron daemon should run the shell scripts. Collection of data in this manner is useful to characterize system usage over a period of time and determine peak usage hours.

You can insert a dummy record into the standard system activity daily data file at the time of system start by un commenting corresponding lines in the /etc/rc script. The sar command reports time change not positive for any record where CPU times are less than the previous record. This occurs if you reboot the system with the dummy record insertion lines in /etc/rc commented out.

Note: The sar command only reports on local activities.

You can use the System application in Web-based System Manager (wsm) to run this command.

You could also use the System Management Interface Tool (SMIT) smit sar fast path to run this command.

Flags

-A	Without the -P flag, using the -A flag is equivalent to specifying -abckmqruvwy. When used with the -P flag, the -A is equivalent to specifying -acmuw.
-a	Reports use of file access system routines specifying how many times per second several of the system file access routines have been called. When used with the -P flag, the information is provided for each specified processor; otherwise, it is provided only system-wide. The following values are displayed: dirblk/s Number of 512-byte blocks read by the directory search routine to locate a directory entry for a specific file. iget/s Calls to any of several i-node lookup routines that support multiple file system types. The iget routines return a pointer to the i-node structure of a file or device. lookuppn/s Calls to the directory search routine that finds the address of a v-node given a path name.
-b	Reports buffer activity for transfers, accesses, and cache (kernel block buffer cache) hit ratios per second. Access to most files in Version 3 bypasses kernel block buffering and therefore does not generate these statistics. However, if a program opens a block device or a raw character device for I/O, traditional access mechanisms are used making the generated statistics meaningful. The following values are displayed: bread/s, bwrit/s Reports the number of block I/O operations. These I/Os are generally performed by the kernel to manage the block buffer cache area, as discussed in the description of the lread/s value. lread/s, lwrit/s Reports the number of logical I/O requests. When a logical read or write to a block device is performed, a logical transfer size of less than a full block size may be requested. The system accesses the physical device units of complete blocks and buffers these blocks in the kernel buffers that have been set aside for this purpose (the block I/O cache area). This cache area is managed by the kernel, so that multiple logical reads and writes to the block device can access previously buffered data from the cache and require no real I/O to the device. Application read and write requests to the block device are reported statistically as logical reads and writes. The block I/O performed by the kernel to the block device in management of the cache area is reported as block reads and block writes. pread/s, pwrit/s Reports the number of I/O operations on raw devices. Requested I/O to raw character devices is not buffered as it is for block devices. The I/O is performed to the device directly. %rcache, %wcache Reports caching effectiveness (cache hit percentage). This percentage is calculated as: [(100)x(lreads - breads)/ (lreads)].
-c	Reports system calls. When used with the -P flag, the information is provided for each specified processor; otherwise, it is provided only system-wide. The following values are displayed: exec/s, fork/s Reports the total number of fork and exec system calls. sread/s, swrit/s Reports the total number of read/write system calls. rchar/s, wchar/s Reports the total number of characters transferred by read/write system calls. scall/s Reports the total number of system calls. Note: The sar command itself can generate a considerable number of reads and writes depending on the interval at which it is run. Run the sar statistics without the workload to understand the sar command's contribution to your total statistics.
-d	Reports activity for each block device (for example, disk or tape drive) with the exception of XDC disks and tape drives. When data is displayed, the device specification dsk is generally used to represent a disk drive. The device specification used to represent a tape drive is machine dependent. The activity data reported is: %busy, avque Reports the portion of time the device was busy servicing a transfer request, average number of requests outstanding during that time. read/s, write/s, blks/s Reports the number of read/write transfers from or to a device, number of bytes is transferred in 512-byte units. avseek Reports the number of milliseconds per average seek.
-e hh[:mm[:ss]]	Sets the ending time of the report. The default ending time is 18:00.
-f File	Extracts records from File (created by -o File flag). The default value of the File parameter is the current daily data file, the /var/adm/sa/sadd file.
-i Seconds	Selects data records at seconds as close as possible to the number specified by the Seconds parameter. Otherwise, the sar command reports all seconds found in the data file.
-k	Reports kernel process activity. The following values are displayed: kexit/s Reports the number of kernel processes terminating per second. kproc-ov/s Reports the number of times kernel processes could not be created because of enforcement of process threshold limit. ksched/s Reports the number of kernel processes assigned to tasks per second.
-m	Reports message (sending and receiving) and semaphore (creating, using, or destroying) activities per second. When used with the -P flag, the information is provided for each specified processor; otherwise, it is provided only system-wide. The following values are displayed: msg/s Reports the number of IPC message primitives. sema/s Reports the number of IPC semaphore primitives.
-o File	Saves the readings in the file in binary form. Each reading is in a separate record and each record contains a tag identifying the time of the reading.
-P ProcessorIdentifier, ... \| ALL	Reports per-processor statistics for the specified processor or processors. Specifying the ALL keyword reports statistics for each individual processor, and globally for all processors . Of the flags which specify the statistics to be reported, only the -a, -c, -m, -u, and -w flags are meaningful with the -P flag.
-q	Reports queue statistics. The following values are displayed: runq-sz Reports the average number of kernel threads in the run queue. %runocc Reports the percentage of the time the run queue is occupied. swpq-sz Reports the average number of kernel threads waiting to be paged in. %swpocc Reports the percentage of the time the swap queue is occupied. Note: A blank value in any column indicates that the associated queue is empty.
-r	Reports paging statistics. The following values are displayed: cycle/s Reports the number of page replacement cycles per second. fault/s Reports the number of page faults per second. This is not a count of page faults that generate I/O, because some page faults can be resolved without I/O. slots Reports the number of free pages on the paging spaces. odio/s Reports the number of non paging disk I/Os per second.
-s hh[:mm[:ss]]	Sets the starting time of the data, causing the sar command to extract records time-tagged at, or following, the time specified. The default starting time is 08:00.
-u	Reports per processor or system-wide statistics. When used with the -P flag, the information is provided for each specified processor; otherwise, it is provided only system-wide. Because the -u flag information is expressed as percentages, the system-wide information is simply the average of each individual processor's statistics. Also, the I/O wait state is defined system-wide and not per processor. The following values are displayed: %idle Reports the percentage of time the cpu or cpus were idle with no outstanding disk I/O requests. %sys Reports the percentage of time the cpu or cpus spent in execution at the system (or kernel) level. %usr Reports the percentage of time the cpu or cpus spent in execution at the user (or application) level. %wio Reports the percentage of time the cpu(s) were idle during which the system had outstanding disk/NFS I/O request(s). See detailed description above. Note: The sar command reports system unit activity if no other specific content options are requested.
-V	Reads the sar files created on previous versions of the operating system. This flag can only be used with the -f flag.
-v	Reports status of the process, kernel-thread, i-node, and file tables. The following values are displayed: file-sz, inod-sz, proc-sz , thrd-sz Reports the number of entries in use for each table.
-w	Reports system switching activity. When used with the -P flag, the information is provided for each specified processor; otherwise, it is provided only system-wide. The following value is displayed: pswch/s Reports the number of context switches per second.
-y	Reports tty device activity per second. canch/s Reports tty canonical input queue characters. This field is always 0 (zero) for AIX Version 4 and later versions. mdmin/s Reports tty modem interrupts. outch/s Reports tty output queue characters. rawch/s Reports tty input queue characters. revin/s Reports tty receive interrupts. xmtin/s Reports tty transmit interrupts.

Security

Access Control: These commands should grant execute (x) access only to members of the adm group.

Examples

To report system unit activity, enter:
```
sar
```
To report current tty activity for each 2 seconds for the next 20 seconds, enter:
sar -y -r 2 20
To watch system unit for 10 minutes and sort data, enter:
sar -o temp 60 10
To report cpu activity for the first two processors, enter:
sar -u -P 0,1
This produces output similar to the following:
```
cpu  %usr  %sys  %wio  %idle
0      45    45     5      5
1      27    65     3      5
```

To report message, semaphore, and cpu activity for all processors and system-wide, enter:
sar -mu -P ALL
On a four-processor system, this produces output similar to the following (the last line indicates system-wide statistics for all processors):

cpu  msgs/s  sema/s  %usr  %sys  %wio  %idle
0      7       2       45    45     5     5
1      5       0       27    65     3     5
2      3       0       55    40     1     4
3      4       1       48    41     4     7
-     19       3       44    48     3     5

To read the system activity file called File generated on previous versions of the operating system, enter:
sar -V -f File

Files

/usr/sbin/sar	Contains the sar command.
/bin/sar	Indicates the symbolic link to the sar command.
/var/adm/sa/sadd	Indicates the daily data file, where the dd parameter is a number representing the day of the month.

Related Information

The sadc command, sa1 command, sa2 command.

Accounting Overview in the AIX 5L Version 5.1 System Management Concepts: Operating System and Devices.

Setting Up an Accounting System in the AIX 5L Version 5.1 System Management Guide: Operating System and Devices.

Accounting Commands Overview in the AIX 5L Version 5.1 System Management Concepts: Operating System and Devices lists accounting commands that can be run automatically or entered from the keyboard.

For information on installing the Web-based System Manager, see Chapter 2: Installation and System Requirements in AIX 5L Version 5.1 Web-based System Manager Administration Guide.