An accurate and complete definition of the system's workload is critical to predicting or understanding its performance. A difference in workload can cause far more variation in the measured performance of a system than differences in CPU clock speed or random access memory (RAM) size. The workload definition must include not only the type and rate of requests to the system, but also the exact software packages and in-house application programs to be executed.
Whenever possible, observe the current users of existing applications to get authentic, real-world measurements of the rates at which users interact with their workstations or terminals.
Make sure that you include the work that your system is doing "under the covers." For example, if your system contains file systems that are NFS-mounted and frequently accessed by other systems, handling those accesses is probably a significant fraction of the overall workload, even though your system is not officially a server.
A benchmark is a workload that has been standardized to allow comparisons among dissimilar systems. Any benchmark that has been in existence long enough to become industry-standard has been studied exhaustively by systems developers. Operating systems, compilers, and in some cases hardware, have been tuned to run the benchmark with lightning speed.
Unfortunately, few real workloads duplicate the exact algorithms and environment of a benchmark. Even those industry-standard benchmarks that were originally derived from real applications might have been simplified and homogenized to make them portable to a wide variety of hardware platforms. The environment in which they run has been constrained in the interests of reproducible measurements.
Any reasoning similar to "System A is rated at 50 percent more MegaThings than System B, so System A should run my program 50 percent faster than System B" might be a tempting shortcut, but may be inaccurate. There is no benchmark with such universal applicability. The only valid use for industry-standard benchmarks is to narrow the field of candidate systems that will be subjected to a serious evaluation. There is no substitute for developing a clear understanding of your workload and its performance in systems under consideration.
After defining the workload that the system will have to process, you can choose performance criteria and set performance objectives based on those criteria. The main overall performance criteria of computer systems are response time and throughput.
Response time is the elapsed time between when a request is submitted and when the response from that request is returned. Examples include how long a database query takes, or how long it takes to echo characters to the terminal, or how long it takes to access a Web page.
Throughput is a measure of the amount of work over a period of time. In other words, it is the number of workload operations that can be accomplished per unit of time. Examples include database transactions per minute, kilobytes of a file transferred per second, kilobytes of a file read or written per second, or Web server hits per minute.
The relationship between these metrics is complex. In some cases, you might have to trade off one against the other. In other situations, a single change can improve both. Sometimes you can have higher throughput at the cost of response time or better response time at the cost of throughput. Acceptable performance is based on reasonable throughput combined with reasonable response time.
In planning for or tuning any system, make sure that you have clear objectives for both response time and throughput when processing the specified workload. Otherwise, you risk spending analysis time and resource dollars improving an aspect of system performance that is of secondary importance.