 ### Using "iostat" to Determine IO Rate, Block Size

#### Audience: All

Date: April 22, 2002

Last weeks tip (http://www.aixtips.com/AIXtip/sizing_disk_for_io.htm ) showed a methodology for sizing the minimum number of disks to support a given transaction rate (IO per second)**.

The iostat command can be used to get the data for the sizing methodology. Here's how to calculate the statistics by disk:

I/O Rate = tps
Block size = kbps / tbps
%Write = ( KB_write )/( KB_read + KB_write )
Sample Interval = ( KB_read + KB_write )/ kbps

The statistics have to be summed over all disks. The attached "analyze_iostat" perl script does this summation. It can be run either as a pipe, or against a saved file. For example:

```Pipe:               iostat 5 10 | analyze_iostat
Saved file:     analyze_iostat
```

The analyze_iostat output list overall statistics for each interval, as well as an average for the entire sample. The following is a sample of the output.

```                      |--KB_Read---| |--KB_Write--|  AvBlkSz Interval
Count   kbps      TPS   % = R/(R+W)    % = W/(R+W)    (kb)   (sec)
-------------------------------------------------------------------------
1     4635.9   1159.0   26332 (100%)       0 (  0%)    4.0      6
2     4252.6   1062.8   24204 (100%)     121 (  0%)    4.0      6
3     4638.0   1159.5   26344 (100%)       0 (  0%)    4.0      6
4     4630.6   1157.6   26348 (100%)       0 (  0%)    4.0      6
5     4956.2    250.7    3492 ( 12%)   24560 ( 88%)   19.8      6
6     9770.6    179.9     172 (  0%)   55716 (100%)   54.3      6
7     5144.4    132.4    2032 (  7%)   27188 ( 93%)   38.9      6
8     2040.6     94.2    6196 ( 53%)    5456 ( 47%)   21.7      6
9     1914.2     92.1    5824 ( 53%)    5068 ( 47%)   20.8      6
10    2251.9     93.0    6676 ( 52%)    6160 ( 48%)   24.2      6
11    2289.8     88.2    6692 ( 52%)    6268 ( 48%)   26.0      6
-------------------------------------------------------------------------
Avg   4229.5    497.2   12210 ( 51%)   11867 ( 49%)   20.1      6
-------------------------------------------------------------------------
```

One caution. This script measures only physical I/O. If the disks are mirrored, there are two physical writes for each logical write. So, for example, if you are moving from mirrored to Raid 5, convert the KB_write and tps columns to logical writes.

(Observation: AIX 5.1.0.10 has a bug in the iostat output. To fix, update to current levels)

Perl script to analyze iostat data:

```#!/usr/bin/perl
# Bruce Spencer IBM, 12apr02
# Purpose: estimate AIX block size, tps rate, and %read/%write
\$VERSION="1.0, April 12, 2002";

# If you want to list disks that exceed a threshold utilization
# set the next value to be greater than 0.
# Turn off by setting to 0
\$THRESHOLD_DISK_UTIL=40;

# Help screen
if ( \$ARGV eq "-h") {
\$PRGM=\$0;
system("tput clear");
print("Program name:  \$PRGM, Version \$VERSION \n\n");
print("Purpose:  Analyze iostat output to estimate I/O block size, transactions/sec, read/write content.\n\n");
print("Usage:  \$PRGM [iostat file]\n\n\n");
print("Example 1: Input from file.\n");
print("\n\tiostat 10 30 > iostat.out\n");
print("\t\$PRGM iostat.out \n\n");
print("Example 2: Input from stdin.\n");
print("\n\tiostat 10 30 \| \$PRGM\n\n");
print("Note: Be sure there is more than one iostat interval in the sample.\n");
print("This program skips the first interval (historic data).\n\n");
exit(0);
} # end if

# Determine if input is stdin or file
if (@ARGV == 0) {
#No file specified, assume stdin
\$INFILE="<&0";
\$|=1; # Don't buffer input
} else{
\$INFILE=\$ARGV;
} # end if

# Open file or stdin
unless (open(INPUT, \$INFILE)) { die ("File open error\n");}

# Debug
# unless (open(INPUT, "iostat.out")) { die ("File open error\n");}

# Read the iostat data for each interval
# 1. Data for each interval starts after line "Disks:  %tm_act....."
# 2. Read interval data. Output interval stats. Accumulate overall stats.
# 3. Interval data ends with "^cd0" or eof.

write;
\$~="LINE";
write;

# Switch to column format for output
\$~="COLUMNS";

# Skip first iostat interval because it is historic data
&next_interval;

while ( 1 ) {

&next_interval;

# Zero counters for this interval
&zerocounters;

# Check for end of interval (CDROM or empty line)
if (\$input =~ /^cd/ || \$input eq "" ) {last;}

# Parse the line for I/O data
@cols =	split(/\s+/,\$input);
\$count++;
\$sum_kbps+=\$cols;
\$sum_tps+=\$cols;
\$sum_write+=\$cols;
if ( \$cols > 0 ) {
\$sum_blocksize+=\$cols/\$cols;
\$blocksize_count++;
} #end if
\$disk_util{\$cols}+= \$cols;

} # end while

# Summarize the I/O data for this interval
&summarize;

}# end while

close(INPUT);

##############################################################
#  Subroutines
##############################################################

# Calculate grand totals
sub grandtotals() {

# Calculate the overall statistics
# Reuse interval variables so we can use the write function

# Set print format
\$~="COLUMNS";

if (\$interval >0) {

\$sum_kbps = \$kbps_overall/\$interval;
\$sum_tps  = \$tps_overall/\$interval;
\$sum_write= \$write_overall/\$interval;

if ( \$sum >0 ) {
} else {
\$pct_write="";
}

if (\$block_overall_cnt >0 ){
\$blocksize=\$block_overall/\$block_overall_cnt;
} else {
\$blocksize="";
} # end if

#		if ( \$tps_overall > 0 ) {
#			\$time_int= \$sum/\$tps_overall;
#		} else {
#			\$time_int="";
#		} # end if

# Save the interval count in "count"
\$count=\$interval;
\$interval="Avg";

# Print horizontal line
\$~="LINE";
write;

# Print the overall averages
\$~="COLUMNS";
write;

} #end if

# Print out disk that exceeded threshold
if (\$count > 0 && \$THRESHOLD_DISK_UTIL > 0 ) {

# Print horizontal line
\$~="LINE";
write;

write;

# Set print format for threshold
\$~="THRESHOLD";

foreach \$disk ( keys %disk_util ) {
\$disk_util{\$disk}/=\$count;
if ( \$disk_util{\$disk} > \$THRESHOLD_DISK_UTIL ) {
write;
} # end if
} # end foreach
} # end if

} # end grandtotals

# Summarizes the interval statistics
sub summarize() {

if ( \$count < 1  ) {
print ("Input error: Disk count equals zero. Exiting\n");
exit;
} # end if

# Interval statistics

if ( \$sum > 0 ) {
} else {
\$pct_write=" ";
}

# Sample Interval (seconds)
if (\$sum_kbps > 0) {
\$time_int= \$sum/\$sum_kbps;
} else {
\$time_int="";
}

if (\$blocksize_count == 0) {
\$blocksize=" ";
} else {
\$blocksize=\$sum_blocksize/\$blocksize_count;
} # end if

# Sum statistics for global sample
\$interval++;
\$kbps_overall+=\$sum_kbps;
\$tps_overall+=\$sum_tps;
\$write_overall+=\$sum_write;

if (\$blocksize_count >0) {
\$block_overall += \$blocksize;
\$block_overall_cnt++;
} # end if

# Print out interval summary
write;

} #end summarize

# Zero out interval counters
sub zerocounters() {
\$count=0;
\$sum_tps=0;
\$sum_write=0;
\$sum_blocksize=0;
\$blocksize_count=0;
\$sum_kbps=0;
} # end zerocounters

# Increment line to next iostat interval
sub next_interval {

#Increment to beginning of next iostat interval
do {
} until (\$input =~ /^Disks/);

} # end next_interval

\$input=<INPUT> ;

# check for end of file
if ( eof ) {
&summarize;
&grandtotals;
exit;
} #end if

chop(\$input);

##############################################################
#  Print formats
##############################################################

Count   kbps      TPS   % = R/(R+W)    % = W/(R+W)    (kb)   (sec)
.

format COLUMNS =
@<<< @####.#  @####.# @###### (@##%) @###### (@##%) @###.#   @###.#
.

format LINE =
-------------------------------------------------------------------------
.

The following disks exceed the threshold @###.# % utilization.
\$THRESHOLD_DISK_UTIL

.

format THRESHOLD =
@<<<<<<<<<<   @###.#
\$disk, \$disk_util{\$disk}
.
```

Bruce Spencer,
baspence@us.ibm.com