View figure.
This section contains some hints, tips and other information to help in tuning the SP system. These sections provide specific information on the SP and its subsystems. By no means is this section complete and comprehensive, but it addresses some SP-specific considerations. See the book AIX Version 4 Performance Tuning Guide for additional AIX tuning information.
When configuring larger systems, you need to consider several topics when setting up your network. These are the SP Ethernet, the outside network connections, the routers, the gateways, and the switch traffic.
The SP Ethernet is the network that connects a control workstation to each of the nodes in the SP that are to be operated and managed by that control workstation using PSSP. When configuring the SP Ethernet, the most important consideration is the number of subnets you configure. Because of the limitation on the number of simultaneous network installs, the routing through the SP Ethernet can be complicated. Usually the amount of traffic on this network is low.
If you connect the SP Ethernet to your external network, you must make sure that the user traffic does not overload the SP Ethernet network. If your outside network is a high speed network like FDDI or HIPPI, routing the traffic to the SP Ethernet can overload it. For gateways to FDDI and other high speed networks, you should route traffic over the switch network. You should configure routers or gateways to distribute the network traffic so that one network or subnet is not a bottleneck. If the SP Ethernet is overloaded by user traffic, move the user traffic to another network.
If you expect a lot of traffic, then you should configure several gateways. You can monitor all the traffic on these networks using the standard network monitoring tools. For more information on these tools, refer to the AIX Version 4 Performance Tuning Guide publication.
When planning your SP Ethernet admin LAN topology, consider your network install server requirements. The network install process uses the SP Ethernet for transferring the install image from the install server to the SP nodes. Running lots of concurrent network installs can exceed the capacity of the SP Ethernet admin LAN. The following are suggested guidelines for designing the SP Ethernet topology for efficient network installs. Many of the configuration options will require additional network hardware beyond the minimal node and control workstation requirements. There are also network addressing and security issues to consider.
|The p690 server gets network connected and does not require you to |use en0. For all other nodes, you must use the en0 adapter to connect each node to the SP Ethernet admin LAN. The following requirements pertaining to the SP Ethernet admin LAN exist for all configurations:
For small systems, you can use the control workstation as the network install server. This means that the SP Ethernet admin LAN is a single network connecting all nodes to the control workstation. When installing the nodes, limit yourself to installing eight nodes at a time because that is the limit of acceptable throughput on the Ethernet. Figure 7 shows an Ethernet topology for a single-frame system.
Figure 7. Ethernet topology with one adapter for a single-frame SP system
View figure.
An alternate way to configure your system is to install a second Ethernet adapter in your control workstation, if you have an available I/O slot, and use two Ethernet segments to the SP nodes. Connect each network to half of the SP nodes. When network installing the frame, you can install all 16 nodes at the same time. Figure 8 shows this alternate Ethernet topology for a single-frame system.
Figure 8. Ethernet topology with two adapters for single-frame SP system
View figure.
Set up your SP Ethernet admin LAN routing so nodes on one Ethernet can communicate to nodes on the other network. Set up your network mask so that each SP Ethernet is its own subnet within a larger network address. Consult your local network administrator about getting and assigning network addresses and network masks.
For multiple frame systems, you might want to spread the network traffic over multiple Ethernets, and keep the maximum number of simultaneous installs per network to eight. You can use the control workstation to network install specific SP nodes which will be the network install servers for the rest of nodes.
Following are three ways to accomplish this.
Figure 9. Method 1 Ethernet topology for multi-frame SP system
View figure.
Once again, set up your SP Ethernet routing so nodes on one Ethernet can communicate to nodes on another. Set up your network mask so that each SP Ethernet is its own subnet within a larger network address. Consult your local network administrator about getting and assigning network addresses and network masks.
This method is applicable up to the number of slots your control workstation has available.
Figure 10. Method 2 Ethernet topology for multi-frame SP system
View figure.
When using this method, installing the nodes requires that you first install the network install node in each frame. The second set of installs will install up to eight additional nodes on the frame. The last install, if needed, installs the rest of the nodes in each frame.
Be forewarned that this configuration usually brings performance problems due to two phenomena:
You can improve the performance here by adding an external router, similar to that described in method 3.
Figure 11. Method 3 Ethernet topology for multi-frame SP system
View figure.
An alternative to the router in this configuration is an Ethernet switch, which could have a high-speed network connection to the control workstation.
If your configuration will grow over time to a large configuration, you might want to dedicate your network install nodes in a different manner.
For very large configurations you might want to dedicate a frame of nodes as designated network install nodes, as shown in Figure 12. In this configuration, each SP Ethernet from the control workstation is connected to up to eight network install nodes in a frame. These network install nodes are in turn connected to additional frames.
Figure 12. Boot-server frame approach
View figure.
The advantage of this is that when you add an additional frame to your SP configuration, all you need to do is connect the new frame to one of the network install nodes, and reconfigure the system.
The network install procedure for this system is the same as for multiple frame systems. You first install the network install servers at a rate of eight per SP Ethernet segment. The network install servers then install eight other nodes until all nodes are installed.
The network address usually used for the SP Ethernet is a class C internet address. This address has a limit of 256 individual addresses before you need to add additional network addresses for the SP Ethernet. If your system is expected to grow beyond this number of nodes, you should plan with your Network Administrator additional network addresses for future SP Ethernet expansion. This will save you from having to re-assign the SP Ethernet addresses when you reach the address limit.
Customer application data can be delivered to applications running on the SP from file servers. These file servers can be either internal SP nodes or separate external systems. The location of the data, how often you refer to it, and whether it is accessed in read-only or both read and write modes affect the performance of applications using this data. Applications that have a high data reference rate, especially those that read and write data, benefit from having the data closely located to the node on which the application executes. The co-location of data and applications minimizes the amount of network processing required to move the data to and from its file server.
When planning for home directory servers, you must determine how much traffic will be generated by requests from the nodes to the server. Because some home directories are NFS, AFS, or DFS-mounted, you need to determine the amount of traffic in operations per second.
If the amount of traffic is greater than the capacity of a single network, you need to add additional networks and divide the number of nodes per network to the server. If the amount of traffic is greater than the capacity on the server, you need to configure additional servers, each connected to all networks.
When you install the SP system, you can define one or more authentication servers. Authentication provides a more secure SP system by verifying the identity of clients that access key systems management facilities.
You can install, configure, and use the DCE security services, the PSSP implementation of Kerberos V4 authentication, or you can integrate your SP system into an existing authentication domain, such as a DCE or AFS cell. If you choose to use AFS authentication servers, note in particular the section on the assignment of TCP/IP port numbers in the /etc/services file.
Carefully plan the location of master and replica authentication servers. The following are some considerations:
pSeries or RS/6000 workstations that are isolated by physical location or have limited
network access.
pSeries or RS/6000 node. It might not be good to place the DCE master server on
an SP node but it could be good for a replica server.
pSeries or RS/6000 node.
As of PSSP 3.2, you have more authentication options than before, but any node still running a pre-PSSP 3.2 release requires use of Kerberos V4. See Chapter 6, Planning for security for more options and planning information.
If you need still more information, see the books AIX Version 4 System Management Guide and PSSP: Administration Guide.
The pv_list attribute in the SDR specifies on which disks to create the root volume group (rootvg by default) and to transfer the mksysb image during AIX network installation of a node. The default value of this attribute is hdisk0 with one exception. For the POWER3 SMP high node, the default is the SCSI disk locations of the two internal disks delivered with the node. Also by default for POWER3 SMP high nodes, the two internal disks will be used for mirroring with two copies of the root volume group. If your POWER3 SMP high node is not configured with the two internal disks, you must override the default. Depending on your nodes and environment, you might have the installation include additional hard disks.
You can use more than one disk when the mksysb image is larger than the disk can hold or when you need the root volume group to span multiple disks. If you do not have either of these circumstances, you should not install on more than one disk. If you do use more than one disk, keep in mind that the first disk in a node is not necessarily hdisk0. When you boot up a node, the first disk found is hdisk0. If you have a fast, wide external disk attached to a node, it can come up as hdisk0.
If you have another disk, you can define a different root volume group on that disk and import it. This lets you reinstall the node and import the volume group without having to back up and restore the data on the non-install disk.
|Check your disks to ensure your install image is on internal |disks. With bootable external disk support like SSA or Fibre Channel |disks, you can have the install image on external disk as long as that image |is not shared across nodes. However, IBM suggests you still keep the |install image on internal disk for efficiency.
To see the value of the pv_list attribute, run the splstdata command with the -v option. To change the pv_list attribute, use the spchvgobj command with the -h option. See the book PSSP: Command and Technical Reference for more information on the spchvgobj command. See the book PSSP: Administration Guide for more information about managing root volume groups, including mirroring the root volume group and alternate root volume groups.