The rmss (Reduced-Memory System Simulator) command provides you with a means to simulate different sizes of real memories that are smaller than your actual machine, without having to extract and replace memory boards. Moreover, the rmss command provides a facility to run an application over a range of memory sizes, displaying, for each memory size, performance statistics such as the response time of the application and the amount of paging. The rmss command is designed to help you answer the question: "How many megabytes of real memory does a system need to run the operating system and a given application with an acceptable level of performance?". In the multiuser context, it is designed to help you answer the question: "How many users can run this application simultaneously in a machine with X megabytes of real memory?"
The main use for the rmss command is as a capacity planning tool, to determine how much memory a workload needs. It can also be used as a problem determination tool, particularly for those cases where having more memory degrades performance.
To determine whether the rmss command is installed and available, run the following command:
# lslpp -lI perfagent.tools
It is important to keep in mind that the memory size simulated by the rmss command is the total size of the machine's real memory, including the memory used by the operating system and any other programs that may be running. It is not the amount of memory used specifically by the application itself. Because of the performance degradation it can cause, the rmss command can be used only by a root user or a member of the system group.
The rmss command can be invoked in two ways:
The first invocation technique is useful when you want to get the look and feel of how your application performs at a given system memory size, when your application is too complex to be expressed as a single command, or when you want to run multiple instances of the application. The second invocation technique is appropriate when you have an application that can be invoked as an executable program or shell script file.
Note: Before using the rmss command, run the command /usr/samples/kernel/schedtune -h 0 to turn off VMM memory-load control (see Tuning VMM Memory Load Control with the schedtune Command). Otherwise, VMM memory-load control may interfere with your measurements at small memory sizes. When your experiments are complete, reset the memory load control parameters to the values that are typically in effect on your system (if you typically use the default parameters, use /usr/samples/kernel/schedtune -D).
To change the memory size and exit, use the -c flag. For example, to change the memory size to 128 MB, use the following:
# rmss -c 128
The memory size is an integer or decimal fraction number of megabytes (for example, 128.25). Additionally, the size must be between 8 MB and the amount of physical real memory in your machine. Depending on the hardware and software configuration, the rmss command may not be able to change the memory size to small sizes, because of the size of inherent system structures such as the kernel. When the rmss command is unable to change to a given memory size, it displays an error message.
The rmss command reduces the effective memory size of a system by stealing free page frames from the list of free frames that is maintained by the VMM. The stolen frames are kept in a pool of unusable frames and are returned to the free frame list when the effective memory size is to be increased. Also, the rmss command dynamically adjusts certain system variables and data structures that must be kept proportional to the effective size of memory.
It may take a short while (up to 15 to 20 seconds) to change the memory size. In general, the more you want to reduce the memory size, the longer the rmss command takes to complete. When successful, the rmss command responds with the following message:
Simulated memory size changed to 128.00 Mb.
To display the current memory size, use the -p flag, as follows:
# rmss -p
The rmss output is as follows:
Simulated memory size is 128.00 Mb.
Finally, if you want to reset the memory size to the actual memory size of the machine, use the -r flag, as follows:
# rmss -r
No matter what the current simulated memory size, using the -r flag sets the memory size to be the physical real memory size of the machine. A side effect of the rmss -r command is that the related vmtune parameters are also reset to their corresponding defaults.
Because this example was run on a 256 MB machine, the rmss command responded as follows:
Simulated memory size changed to 256.00 Mb.
Note: The rmss command reports usable real memory. On machines that contain bad memory or memory that is in use, the rmss command reports the amount of real memory as the amount of physical real memory minus the memory that is bad or in use by the system. For example, the rmss -r command might report:
Simulated memory size changed to 79.9062 Mb.This could be a result of some pages being marked bad or a result of a device that is reserving some pages for its own use (and thus not available to the user).
The -c, -p and -r flags of the rmss command have an advantage over the other options in that they allow you to experiment with complex applications that cannot be expressed as a single executable program or shell script file. On the other hand, the -c, -p, and -r options have a disadvantage in that they force you to do your own performance measurements. Fortunately, you can use the command vmstat -s to measure the paging-space activity that occurred while your application ran.
By running the command vmstat -s, running your application, then running the command vmstat -s again, and subtracting the number of paging-space page-ins before from the number of paging-space page-ins after, you can determine the number of paging-space page-ins that occurred while your program ran. Furthermore, by timing your program, and dividing the number of paging-space page-ins by the program's elapsed run time, you can obtain the average paging-space page-in rate.
It is also important to run the application multiple times at each memory size, for two reasons:
To summarize, consider the following set of steps as a desirable way to invoke the rmss command:
while there are interesting memory sizes to investigate:
{
change to an interesting memory size using rmss -c;
run the application once as a warm-up;
for a couple of iterations:
{
use vmstat -s to get the "before" value of paging-space page ins;
run the application, while timing it;
use vmstat -s to get the "after" value of paging-space page ins;
subtract the "before" value from the "after" value to get the
number of page ins that occurred while the application ran;
divide the number of paging-space page ins by the response time
to get the paging-space page-in rate;
}
}
run rmss -r to restore the system to normal memory size (or reboot)
The calculation of the (after - before) paging I/O numbers can be automated by using the vmstatit script described in Determining Whether the Problem is Related to Disk or Memory.
The -s, -f, -d, -n, and -o flags are used in combination to invoke the rmss command as a driver program. As a driver program, the rmss command executes a specified application over a range of memory sizes and displays statistics describing the application's performance at each memory size. The syntax for this invocation style of the rmss command is as follows:
rmss [ -s smemsize ] [ -f fmemsize ] [ -d memdelta ]
[ -n numiterations ] [ -o outputfile ] command
Each of the following flags is discussed in detail below. The -s, -f, and -d flags are used to specify the range of memory sizes.
All values are in integer or decimal fractions of megabytes. For example, if you wanted to run and measure a command at sizes 256, 224, 192, 160 and 128 MB, you would use the following combination:
-s 256 -f 128 -d 32
Likewise, if you wanted to run and measure a command at 128, 160, 192, 224, and 256 MB, you would use the following combination:
-s 128 -f 256 -d 32
If the -s flag is omitted, the rmss command starts at the actual memory size of the machine. If the -f flag is omitted, the rmss command finishes at 8 MB. If the -d flag is omitted, there is a default of 8 MB between memory sizes.
What values should you choose for the -s, -f, and -d flags? A simple choice would be to cover the memory sizes of systems that are being considered to run the application you are measuring. However, increments of less than 8 MB can be useful, because you can get an estimate of how much space you will have when you settle on a given size. For instance, if a given application thrashes at 120 MB but runs without page-ins at 128 MB, it would be useful to know where within the 120 to 128 MB range the application starts thrashing. If it starts at 127 MB, you may want to consider configuring the system with more than 128 MB of memory, or you may want to try to modify the application so that there is more space. On the other hand, if the thrashing starts at 121 MB, you know that you have enough space with a 128 MB machine.
The -n flag is used to specify how many times to run and measure the command at each memory size. After running and measuring the command the specified number of times, the rmss command displays statistics describing the average performance of the application at that memory size. To run the command 3 times at each memory size, you would use the following:
-n 3
If the -n flag is omitted, the rmss command determines during initialization how many times your application must be run to accumulate a total run time of 10 seconds. The rmss command does this to ensure that the performance statistics for short-running programs will not be significantly skewed by outside influences, such as daemons.
Note: If you are measuring a very brief program, the number of iterations required to accumulate 10 seconds of CPU time can be very large. Because each execution of the program takes a minimum of about 2 elapsed seconds of rmss overhead, specify the -n parameter explicitly for short programs.
What are good values to use for the -n flag? If you know that your application takes much more than 10 seconds to run, you can specify -n 1 so that the command is run twce, but measured only once at each memory size. The advantage of using the -n flag is that the rmss command will finish sooner because it will not have to spend time during initialization to determine how many times to run your program. This can be particularly valuable when the command being measured is long-running and interactive.
It is important to note that the rmss command always runs the command once at each memory size as a warm-up before running and measuring the command. The warm-up is needed to avoid the I/O that occurs when the application is not already in memory. Although such I/O does affect performance, it is not necessarily due to a lack of real memory. The warm-up run is not included in the number of iterations specified by the -n flag.
The -o flag is used to specify a file into which to write the rmss report. If the -o flag is omitted, the report is written into the file rmss.out.
Finally, command is used to specify the application to be measured. It can be an executable program or shell script, with or without command-line arguments. There are some limitations on the form of the command however. First, it cannot contain the redirection of input or output (for example, foo > output or foo < input). This is because the rmss command treats everything to the right of the command name as an argument to the command. To redirect, place the command in a shell script file.
Usually, if you want to store the rmss output in a specific file, use the -o option. If you want to redirect the standard output of the rmss command (for example, to concatenate it to the end of an existing file) then use the Korn shell to enclose the rmss invocation in parentheses, as follows:
# (rmss -s 24 -f 8 foo) >> output
The Report Generated for the foo Program example was produced by running the rmss command on an actual application program, although the name of the program has been changed to foo for anonymity. The specific command that would have been used to generate the report is as follows:
# rmss -s 16 -f 8 -d 1 -n 1 -o rmss.out foo
Hostname: widgeon.austin.ibm.com Real memory size: 16.00 Mb Time of day: Thu Jan 6 19:04:04 2000 Command: foo Simulated memory size initialized to 16.00 Mb. Number of iterations per memory size = 1 warm-up + 1 measured = 2. Memory size Avg. Pageins Avg. Response Time Avg. Pagein Rate (megabytes) (sec.) (pageins / sec.) ----------------------------------------------------------------- 16.00 115.0 123.9 0.9 15.00 112.0 125.1 0.9 14.00 179.0 126.2 1.4 13.00 81.0 125.7 0.6 12.00 403.0 132.0 3.1 11.00 855.0 141.5 6.0 10.00 1161.0 146.8 7.9 9.00 1529.0 161.3 9.5 8.00 2931.0 202.5 14.5
The report consists of four columns. The leftmost column gives the memory size, while the Avg. Pageins column gives the average number of page-ins that occurred when the application was run at that memory size. It is important to note that the Avg. Pageins column refers to all page-in operations, including code, data, and file reads, from all programs, that completed while the application ran. The Avg. Response Time column gives the average amount of time it took the application to complete, while the Avg. Pagein Rate column gives the average rate of page-ins.
Concentrate on the Avg. Pagein Rate column. From 16 MB to 13 MB, the page-in rate is relatively small (< 1.5 page-ins per second). However, from 13 MB to 8 MB, the page-in rate grows gradually at first, and then rapidly as 8 MB is reached. The Avg. Response Time column has a similar shape: relatively flat at first, then increasing gradually, and finally increasing rapidly as the memory size is decreased to 8 MB.
Here, the page-in rate actually decreases when the memory size changes from 14 MB (1.4 page-ins per second) to 13 MB (0.6 page-ins per second). This is not cause for alarm. In an actual system, it is impossible to expect the results to be perfectly smooth. The important point is that the page-in rate is relatively low at both 14 MB and 13 MB.
Finally, you can make a couple of deductions from the report. First, if the performance of the application is deemed unacceptable at 8 MB (as it probably would be), then adding memory would enhance performance significantly. Note that the response time rises from approximately 124 seconds at 16 MB to 202 seconds at 8 MB, an increase of 63 percent. On the other hand, if the performance is deemed unacceptable at 16 MB, adding memory will not enhance performance much, because page-ins do not slow the program appreciably at 16 MB.
The following example illustrates a report that was generated (on a client machine) by running the rmss command on a command that copied a 16 MB file from a remote (server) machine through NFS.
Hostname: xray.austin.ibm.com Real memory size: 48.00 Mb Time of day: Mon Jan 10 18:16:42 2000 Command: cp /mnt/a16Mfile /dev/null Simulated memory size initialized to 48.00 Mb. Number of iterations per memory size = 1 warm-up + 4 measured = 5. Memory size Avg. Pageins Avg. Response Time Avg. Pagein Rate (megabytes) (sec.) (pageins / sec.) ----------------------------------------------------------------- 48.00 0.0 2.7 0.0 40.00 0.0 2.7 0.0 32.00 0.0 2.7 0.0 24.00 1520.8 26.9 56.6 16.00 4104.2 67.5 60.8 8.00 4106.8 66.9 61.4
Note that the response time and page-in rate in this report start relatively low, rapidly increase at a memory size of 24 MB, and then reach a plateau at 16 and 8 MB. This report shows the importance of choosing a wide range of memory sizes when you use the rmss command. If this user had only looked at memory sizes from 24 MB to 8 MB, he or she might have missed an opportunity to configure the system with enough memory to accommodate the application without page-ins.
One helpful feature of the rmss command, when used in this way, is that it can be terminated (by the interrupt key, Ctrl-C by default) without destroying the report that has been written to the output file. In addition to writing the report to the output file, this causes the rmss command to reset the memory size to the physical memory size of the machine.
You can run the rmss command in the background, even after you have logged out, by using the nohup command. To do this, precede the rmss command by the nohup command, and follow the entire command with an & (ampersand), as follows:
# nohup rmss -s 48 -f 8 -o foo.out foo &
No matter which rmss invocation style you are using, it is important to re-create the end-user environment as closely as possible. For instance, are you using the same model CPU, the same model disks, the same network? Will the users have application files mounted from a remote node via NFS or some other distributed file system? This last point is particularly important, because pages from remote files are treated differently by the VMM than pages from local files.
Likewise, it is best to eliminate any system activity that is not related to the desired system configuration or the application you are measuring. For instance, you do not want to have people working on the same machine as the rmss command unless they are running part of the workload you are measuring.
Note: You cannot run multiple invocations of the rmss command simultaneously.
When you have completed all runs of the rmss command, it is best to shut down and reboot the system. This will remove all changes that the rmss command has made to the system and will restore the VMM memory load control parameters to their typical settings.