WLM monitors and regulates the resource utilization, on a per-class basis, of the threads and processes active on the system. You can set minimum or maximum limits per class for each resource type managed by WLM, as well as a target value per class for each resource. This target is representative of the amount of the resource that is optimal for the jobs in the class.
The shares and limits at the superclass level refer to the total amount of each resource available on the system. At the subclass level, shares and limits refer to the amount of each resource made available to the superclass that the subclass is in (superclass target). The hierarchy of classes is a way to divide the system resources between groups of users (superclasses) and delegate the administration of this share of the resources to superclass administrators. Each superclass administrator can then redistribute this amount of resources between the users in the group by creating subclasses and defining resource entitlements for these subclasses.
WLM manages three types of resources on a percentage consumption basis:
| CPU utilization of the threads in a class | This is the sum of all the CPU cycles consumed by every thread in the class. |
| Physical memory utilization for the processes in a class | This is the sum of all the memory pages which belong to the processes in the class. |
| Disk I/O bandwidth for the class | This is the bandwidth (in 512-byte blocks per second) of all the I/Os started by threads in the class on each disk device accessed by the class. |
Every second, WLM calculates the per-class utilization for each resource during the last second, as a percentage of the total resource available, as follows:
To get more accurate and reliable percentages of per-class disk utilization, WLM uses the statistics provided by the disk drivers (displayed using the iostat command) giving for each disk device the percentage of time that the device has been busy during the last second. WLM counts the number of total blocks which have been read or written on a device during the last second by all the classes accessing the device, and how many blocks have been read or written by each class and what was the percentage of utilization of the device. WLM then calculates the percentage of the disk throughput consumed by each class.
For instance, if the total number of blocks read or written during the last second was 1000 and the device had been 70% busy, this means that a class reading or writing 100 blocks used 7% of the disk bandwidth. Similarly to the CPU time (another renewable resource), the values used by WLM for its disk I/O regulation are also a decayed average over a few seconds of these per second percentages.
For the disk I/O resource, the shares and limits apply to each disk device individually accessed by the class. The regulation is done independently for each device. This means that a class could be over its entitlement on one device, and the I/Os to this device will be regulated while it is under its entitlement on another disk and the I/Os to this other device will not be constrained.
In AIX 5.2 and later, WLM supports accounting and regulation of resources on the basis of total consumption. There are two types of resources that can be regulated in this manner: class totals and process totals.
The target (or desired) resource consumption percentage for a class is determined by the number of shares it has for a particular resource. The shares represent how much of a particular resource a class should get, relative to the other classes in its tier. A class's target percentage for a particular resource is simply its number of shares divided by the number of active shares in its tier. If limits are also being used, the target is limited to the range [minimum, soft maximum]. If the calculated target outside this range, it is set to the appropriate upper/lower bound (see Resource Limits). The number of active shares is the total number of shares of all classes that have at least one active process in them. Since the number of active shares is dynamic, so is the target. If a class is the only active class in a tier, its target will be 100% of the amount of resource available to the tier.
For example, assume three active superclasses classes
in tier 0--A, B, and C--with shares for a particular resource of
15, 10, and 5, respectively, the targets would be:
target(A) = 15/30 = 50%
target(B) = 10/30 = 33%
target(C) = 5/30 = 17%
If some time later, class B becomes inactive (no active
processes), the targets for class A and C will be automatically adjusted:
target(A) = 15/20 = 75%
target(C) = 5/20 = 25%
As you can see, the shares represent a self-adapting percentage which allow the resources allocated to a class to be evenly distributed to or taken from the other classes when it becomes active/inactive.
To allow a high degree of flexibility, the number of shares for a class can be any number between 1 and 65535. Shares can be specified for superclasses and subclasses. For superclasses, the shares are relative to all other active superclasses in the same tier. For subclasses, the shares are relative to all other active subclasses in the same superclass, in the same tier. The shares for a subclass one superclass have no relationship to the shares for a subclass of another superclass.
In some cases, it might be desirable to make the target for a class independent from the number of active shares. To accomplish this, a value of "-" can be specified for the number of shares. In this case, the class will be unregulated for that resource, meaning that it has no shares, and its target is not dependent on the number of active shares. Its target will be set to (resource available to the tier - the sum of mins for all other classes in the tier). This target, or actual consumption (whichever is lower) is then taken out of what is available to other classes in the same tier.
For example, assume classes A, B, C, and D have shares
for a particular resource of "-", 200, 150, and 100, respectively. All classes
are active, and class A is consuming 50% of the resource:
target(A) = unregulated = 100%
target(B) = 200/450 * available = 44% * 50% = 22%
target(C) = 150/450 * available = 33% * 50% = 17%
target(D) = 100/450 * available = 22% * 50% = 11%
Because class A is unregulated and is consuming 50% of the available resource, the other classes only have 50% available to them, and their targets are calculated based on this percentage. Since class A will always be below its target (100%), it will always have a higher priority than all other classes in the same tier that are at or above their targets (see Understanding Class Priority for more information).
If the number of shares is unspecified, a default value of "-" will be used, and the class will be unregulated for that resource. Note that in the first version of WLM, the default share value, if unspecified, was 1.
The shares are specified per class for all the resource types. Shares are specified in stanzas of the shares file. For example:
shares classname: CPU = 2 memory = 4 diskIO = 3
In addition to using shares to define relative resource entitlements, WLM provides the ability to specify resource limits for a class. Resource limits allow the administrator to have more control over resource allocation. These limits are specified as percentages and are relative to the amount of resource available to the tier that the class is in.
There are three type of limits for percentage-based regulation:
Resource limit values are specified in the resource limit file by resource type within stanzas for each class. The limits are specified as a minimum to soft maximum range, separated by a hyphen (-) with white space ignored. The hard maximum when specified follows the soft maximum, and is separated by a semicolon (;). Each limit value is followed by a percent sign (%).
The following are examples using the rules files:
Limits are specified for all resource types, per class, in stanzas of the limits file. For example:
shares classname: CPU = 0%-50%;80% memory = 10%-30%;50%
In this example, no limits are set for disk I/O. Using the system defaults, this translates to the following:
diskIO = 0%-100%;100%
All of the preceding examples assume that the superclasses and subclasses described do not have the inheritance attribute set to yes. Otherwise, the new processes would simply inherit the superclass or subclass from their parent.
The following are the only constraints that WLM places on resource limit values:
When a class with a hard memory limit has reached this limit and requests more pages, the VMM page replacement algorithm (LRU) is initiated and "steals" pages from the limited class, thereby lowering its number of pages below the hard maximum, before handing out new pages. This behavior is correct, but extra paging activity, which can take place even where there are plenty of free pages available, impacts the general performance of the system. Minimum memory limits for other classes are recommended before imposing a hard memory maximum for any class.
Since classes under their minimum have the highest priority in their tier, the sum of the minimums should be kept to a reasonable level, based on the resource requirements of the other classes in the same tier.
The constraint of the sum of the minimum limits within a tier being less than or equal to 100 means that a class in the highest priority tier is always allowed to get resources up to its minimum limit. WLM does not guarantee that the class will actually reach its minimum limit. This depends on how the processes in the class use their resources and on other limits that are in effect. For example, a class might not reach its minimum CPU entitlement because it cannot get enough memory.
For physical memory, setting a minimum memory limit provides some protection for the memory pages of the class's processes (at least for those in the highest priority tier). A class should not have pages stolen when it is below its minimum limit unless all the active classes are below their minimum limit and one of them requests more pages. A class in the highest tier should never have pages stolen when it is below its minimum. Setting a memory minimum limits for a class of interactive jobs helps make sure that their pages will not all have been stolen between consecutive activations (even when memory is tight) and improves response time.
Beginning with AIX 5.2, the total limits can be specified in the limits file with values and units summarized in the following table:
| Resource | Allowed Units | Default Unit | Maximum Value | Minimum Value |
|---|---|---|---|---|
| totalCPU | s, m, h, d, w | s | 230 - 1s | 10 s |
| totalDiskIO | KB, MB, TB, PB, EB | KB | (263 - 1) * 512/1024 KB | 1 MB |
| totalConnectTime | s, m, h, d, w | s | 263 - 1 s | 5 m |
| totalProcesses | - | - | 263 - 1 | 2 |
| totalThreads | - | - | 263 - 1 | 2 |
| totalLogins | - | - | 263 - 1 | 1 |
An example limits stanza follows:
BadUserClass:
totalCPU = 1m
totalConnectTime = 1h
The total limits can be specified using any value in the above table with the following restrictions:
The total limits can be specified at the superclass and subclass level. When checking the limits, the subclass limit is checked before the superclass limit. If both limits are specified, the lower of the two is enforced. If the specified subclass limit is greater than its associated superclass limit, a warning will be issued when the configuration is loaded, but it will be loaded. This is significant for the class total limits since the limit is absolute (not relative to the superclass) and one subclass could consume all resources available to the superclass. If unspecified, the default value for all total limits is "-" which means no limit. By default, class and process total accounting and regulation will be enabled when WLM is running. The -T [class|proc] option of the wlmcntrl command can be used to disable total accounting and regulation.
WLM allocates resources to classes by giving a priority to each class for each resource. The priority of a class is dynamic and is based on the tier, shares, limits, and the current consumption of the class. At any given time, the highest priority class or classes will be given preferential access to resources. At the highest level, tiers represent non-overlapping ranges of class priority. Classes in tier 0 will always be guaranteed to have a higher priority than classes in tier 1 (unless over hard maximum).
When determining class priority, WLM enforces its constraints with the following precedence (highest to lowest):
Even though both shares and limits can be used for each class and for each resource, results are more predictable if only one or the other is used per class.
WLM uses resource sets (or rsets) to restrict the processes in a given class to a subset of the system's physical resources. In WLM, the physical resources managed are the memory and the processors. A valid resource set is composed of memory and at least one processor.
Using SMIT or Web-based System Manager, a system administrator can define and name resource sets containing a subset of the resources available on the system. Then, using the WLM administration interfaces, root user or a designated superclass administrator can use the name of the resource set as the rset attribute of a WLM class. From then on, every process assigned to this WLM class is dispatched only on one of the processors in the resource set, effectively separating workloads for the CPU resource.
All of the current systems have only one memory domain shared by all the resource sets, so this method does not physically separate workloads in memory.
The rset registry services enable system administrators to define and name resource sets so that they can then be used by other users or applications. To alleviate the risks of name collisions, the registry supports a two-level naming scheme. The name of a resource set is in the form name_space/rset_name. Both the namespace and rset_name can each be 255 characters in length, are case-sensitive, and may contain only uppercase and lowercase letters, numbers, underscores, and periods (.). The namespace of sys is reserved by the operating system and used for rset definitions that represent the resources of the system.
The rset definition names are unique within the registry name space. Adding a new rset definition to the registry using the same name as an existing rset definition causes the existing definition to be replaced with the new definition, given the proper permission and privilege. Only root can create, modify, and delete resource sets and update the in-core rset database, using SMIT or Web-based System Manager.
Each rset definition has an owner (user ID), group (group ID), and access permissions associated with it. These are specified at the time the rset definition is created and exist for the purpose of access control. As is the case for files, separate access permissions exist for the owner, group and others that define whether read and/or write permission has been granted. Read permission allows an rset definition to be retrieved while write permission allows an rset definition to be modified or removed.
System Administrator defined rset definitions are kept in the /etc/rsets stanza file. The format of this file is not described, and users must manipulate rsets through the SMIT or Web-based System Manager interfaces to prevent future potential compatibility problems if the file format is modified. As is the case for WLM class definitions, the rset definitions must be loaded into kernel data structures before they can be used by WLM.
Rather than giving an extensive definition of resource sets and how to create them from basic building blocks called system RADs (Resource Access Domains). For an example that details what is in a resource set and how to create new one, see Creating a Resource Set in AIX 5L Version 5.2 System Management Guide: Operating System and Devices.