This section discusses what you need to know to plan your network configuration. Instructions for completing the remaining system planning worksheets begin in Chapter 2, Defining the system that fits your needs. All the worksheets are summarized in Appendix C, SP system planning worksheets.
You must assign IP addresses and host names for each network connection on each node and on the control workstation in your SP system. This repeats some information contained in Completing the node layout worksheets. This repetition is valuable because of the importance of this information.
Because you probably want to attach the SP system to your site networks, you need to plan how to do this. You need to decide:
You need to ensure that all of the addresses you assign are unique within your site network and within any outside networks to which you are attached, such as the Internet. Also, you need to plan how names and addresses will be resolved on your systems (that is, using DNS name servers, NIS maps, /etc/host files or some other method).
| Note |
|---|
|
All names and addresses of all IP interfaces on your nodes must be resolvable on the control workstation and on independent workstations set up as authentication servers before you install and configure the SP system. The SP system uses only IPv4 addresses. Some PSSP components tolerate IPv6 aliases for IPv4 network addresses but not with DCE, HACMP, HACWS, or an SP switch. For information about the SP system tolerating IPv6 aliases for some IPv4 network addresses, see the appendix on the subject in the book PSSP: Administration Guide. Once you have set the host names and IP addresses on the control workstation, you should not change them. |
Some name resolution facilities let you map multiple IP interfaces to the same host name. For the SP system, IBM suggests that you assign unique host names to each IP interface on your nodes.
You can connect different types of LANs to the SP system but regardless of how many types you use, the LANs fall into the following categories:
You also need to understand the considerations that are relevant to certain hardware or system features as they relate to planning the SP networks.
|The SP Ethernet is the administrative LAN that connects all nodes in |one system running PSSP to the control workstation. It is used for PSSP |installation and communication among the control workstation, boot-install |servers, and other nodes in the network. For p690 servers, it is also |used to connect the HMC to the control workstation for hardware control and |monitoring.
For each node, ensure that the SDR reliable_hostname attribute is identical to the default host name returned by the host command for its SP Ethernet IP addresses. With the exception of the p690 nodes, the PSSP components expect that en0 is the connection from the node to the SP Ethernet admin LAN for installs and other PSSP functions. For example, if the en0 IP address of a node is 129.40.133.75, and 'host 129.40.133.75' shows the default host name is k65n11.ppd.pok.ibm.com, then it also should be the host name set in the reliable_hostname attribute in the SDR.
|For a p690 node, you can use any Ethernet adapter that is supported |for connecting to the SP Ethernet admin LAN and identify it by name or by |physical location. For all other nodes, you must connect the SP Ethernet admin LAN to the Ethernet adapter in the node's lowest hardware slot of all the Ethernet adapters on that node. When a node is network booted, it selects the lowest Ethernet adapter from which to perform the install. This Ethernet adapter must be on the same subnet of an Ethernet adapter on the node's boot-install server. In nodes that have an integrated Ethernet adapter, it is always the lowest Ethernet adapter. Be sure to maintain this relationship when adding Ethernet adapters to a node.
You can attach the SP Ethernet to other site networks and use it for other site-specific functions. You assign all addresses and names used for the SP Ethernet admin LAN.
You can make the connections from the control workstation to the nodes in one of three ways. The method you choose should be one that optimizes network performance for the functions required of the SP Ethernet admin LAN by your site. The three connection methods are:
See System topology considerations for sample configurations illustrating these connection methods.
The SP boot-install servers must be on the same subnet as their clients. In the case of a multi-stage, multi-subnet SP Ethernet admin LAN, the control workstation is the boot-install server for the first node in each frame and those nodes are the boot-install servers for the other nodes in the frames.
|Also, when booting from the network, nodes broadcast their host |request over their SP Ethernet admin LAN interface. Therefore, that |interface must be the Ethernet adapter on the node that is connected to the |boot-install network. For all nodes but those on a p690, it must be |en0.
The SP Ethernet admin LAN can provide a means to connect all nodes and the control workstation to your site networks. However, it is likely that you will want to connect your SP nodes to site networks through other network interfaces. If the SP Ethernet admin LAN is used for other networking purposes, the amount of external traffic must be limited. If too much traffic is generated on the SP Ethernet admin LAN, the administration of the SP nodes might be severely impacted. For example, problems might occur with network installs, diagnostic function, and maintenance mode access. In an extreme case, if too much external traffic occurs, the nodes will hang when broadcasting for the network.
Additional Ethernet, Fiber Distributed Data Interface (FDDI), and token-ring networks can also be configured by the PSSP software. Other network adapters must be configured manually. These connections can provide increased network performance in user file serving and other network related functions. You need to assign all the addresses and names associated with these additional networks.
|You can plan to run PSSP on a system with a firewall. For |instructions, see the document on this subject at the Web address given in Finding documentation on the World Wide Web.
If your SP has a switch and you want to use IP for communications over the switch, each node needs to have an IP address and name assigned for the switch interface, the css0 adapter. |If you plan to use the SP Switch2 with two switch planes, you also |need to have an IP address and name assigned for the css1 adapter |and you have the option to use the ml0 aggregate IP |interface. If hosts outside the SP switch network need to communicate over the switch using IP with nodes in the SP system, those hosts must have a route to the switch network through one of the SP nodes.
If you are not enabling ARP on the switch, specify the switch network subnet mask and the IP address of the starting node. After the first address is selected, subsequent node addresses are based on the switch port number assigned. See Understanding placement and numbering. Unlike all other network interfaces, which can have sets of nodes divided into several different subnets, the |SP Switch IP network must be one contiguous subnet that includes all the nodes in the system. |If you use the SP Switch2 and all nodes are running PSSP 3.4, |you have optional connectivity so some nodes can be left off the |switch.
|If you want to assign your switch IP addresses as you do your other |adapters, you must enable ARP for the css0 adapter and, if you are |using two switch planes, for the css1 adapter. If you enable |ARP for those interfaces, you can use whatever IP addresses you wish, and |those IP addresses do not have to be in the same subnet for the whole |system. They must all be resolvable by the host command on the control |workstation.
All but the simplest SP system configurations will likely include several subnets. Thoughtful use of netmasks in planning your networks can economize on the use of network addresses. See the |relevant edition of the AIX book System Management Guide: Communications and Networks for information about Internet addresses and subnets.
As an example, consider an SP Ethernet, where none of the six subnets making up the SP Ethernet have more than 16 nodes on them. A netmask of 255.255.255.224 provides 30 discrete addresses per subnet, which is the smallest range that is usable in the wiring as shown. Using 255.255.255.224 as a netmask, we can then allocate the address ranges as follows:
In the same example, if we used 255.255.255.0 as our netmask, then we would have to use six separate Class C network addresses to satisfy the same wiring configuration (that is, 129.34.130.x, 129.34.131.x, 129.34.132.x, and so on).
If you plan to use an SP Switch Router and the SP Switch Router Adapter for routing purposes in your environment, the next few paragraphs on using standard nodes as a network router might not be applicable to your SP configuration. However, if you are not using the SP Switch Router, you might be interested in some considerations for using your nodes as network routers.
When planning router nodes on your system, several factors can help determine the number of routers needed and their placement in the SP configuration. The number of routers you need can vary depending on your network type. (In some environments, router nodes might also be called gateway nodes.)
For nodes that use Ethernet or token ring as the routed network, a customer network running at full bandwidth results in a lightly loaded CPU on the router node. For nodes that use FDDI as the customer routed network, a customer network running at or near maximum bandwidth results in high CPU utilization on the router node. For this reason, you should not assign any additional role in the computing environment, such as a node in a parallel job, to a router using FDDI as the customer network. You also should not connect more than one FDDI to a router node.
Applications, such as POE and the Resource Manager, should run on nodes other than FDDI routers. However, Ethernet and token ring gateways can run with these applications.
For systems that use Ethernet or token-ring routers, traffic can be routed through the SP Ethernet but careful monitoring of the SP Ethernet will be needed to prevent traffic coming through the router from impacting other users of the SP Ethernet. For FDDI networks, traffic should be routed across the switch to the destination nodes. The amount of traffic coming in through the FDDI network can be up to 10 times the bandwidth the SP Ethernet can handle.
Information about configuring network adapters and the various network tunables on the nodes is in the book PSSP: Administration Guide.
The SP Switch Router is something you can use in a system with the SP Swtich. It is not supported with the SP Switch2. It is by type an extension node, more specifically a dependent node. The SP Switch Router gives you high speed access to other systems. Without the SP Switch Router, you would need to dedicate a standard node to performing external network router functions. Also, because the SP Switch Router is external to the frame, it does not take up valuable processor space.
The SP Switch Router has two optional sizes. The smaller unit has four internal slots and the larger unit has sixteen. One slot must be occupied by an SP Switch Router Adapter card which provides the SP connection. The other slots can be filled with any combination of network connection cards including the types:
Additional SP Switch Router Adapters are needed for communicating between system partitions and other SP systems. These cards provide switching rates of from 4 to 16 GB per second between the router and the external network.
To attach an extension node to an SP Switch, configuration information must be specified on the control workstation. Communication of switch configuration information between the control workstation and the SP Switch Router takes place over the SP system's administrative Ethernet and requires use of the UDP port number 162 on the control workstation. If this port is in use, a new communication port will have to be configured into both the control workstation and the SNMP agent supporting the extension node.
The SP Switch Router requires PSSP 2.3 or later on both the primary node and the primary backup node. Using the SP Switch Router Adapter, the SP Switch Router can be connected to an SP Switch (8-port or 16-port).
The SP Switch Router Adapter in the SP Switch Router can be attached to an SP Switch to improve throughput of data coming into and going out of the SP system. Each SP Switch Router Adapter in the SP Switch Router requires a valid unused switch port in the SP system. See Choosing a valid port on the SP Switch.
|A clustered enterprise server is any of the servers discussed in Question 8: Which and how many nodes do you need?. It is not mounted in an SP frame and has |no SP frame or node supervisor, though some do have comparable function |enabling hardware control and monitoring. It is directly attached to |the SP Ethernet admin LAN and to the control workstation. The means of |connection differ with the server hardware.
In a clustered enterprise server system configuration you can assign frame numbers in any order. However, if you add SP frames with SP nodes or with SP switches, your system will then be subject to all the rules of an SP system and these clustered enterprise servers become SP-attached servers. Remember that those terms reflect only the system configuration in which the servers participate.
|If you might use the SP Switch2 or the SP Switch, you need to plan |the respective switch network. If you might use the SP Switch, plan your system with suitable frame numbers and switch port numbers in advance so you can expand to an SP system without having to totally reconfigure existing servers. See Switch port numbering for a switchless system. Also read and understand the information regarding SP-attached servers in Understanding placement and numbering.
|An SP-attached server is also any of the servers discussed in Question 8: Which and how many nodes do you need? but configured to be attached to the SP |system.
|If the SP system has the SP Switch or it is switchless, an |SP-attached server requires an available node slot within an SP frame to |reserve a valid unused switch port on the switch in the same SP frame. |An SP-attached server is not supported with an SP Switch-8. You must |connect the server to the SP Switch network with an adapter. That |adapter connects to the valid unused switch port in the SP frame. For a |p690 node, each LPAR attaches to the switch. See Choosing a valid port on the SP Switch.
|If the system has the SP Switch2, an SP-attached server, and each |p690 LPAR, node can connect with an adapter to any available switch port in |the SP frame. If all nodes are running PSSP 3.4, you can leave |some nodes not connected to the switch.
You must assign a frame number to an SP-attached server. Be sure to read and understand the information regarding SP-attached servers in Understanding placement and numbering.
Each SP Switch Router Adapter in the SP Switch Router and each adapter for an SP-attached server requires a valid unused port of an SP Switch in the SP system. A valid unused switch port is a switch port that meets the rules for configuring frames and switches.
There are two basic sets of rules for choosing a valid switch port:
These rules are discussed further in this chapter.
One example of using an empty node slot position is a single frame system with fourteen thin nodes located in slots 1 through 14. This system has two unused node slots in position 15 and 16. These two empty node slots have corresponding switch ports which provide valid connections for an SP Switch Router Adapter |or the adapter of an SP-attached server.
Another example is a logical pair, two frame system with one shared switch. The first frame is fully populated with eight wide nodes. The second frame has three wide nodes in slots 1, 3, and 5 (see later sections in this chapter for explanations of node numbering schemes). The only valid switch ports in this configuration would be those switch ports associated with node slots 7, 9,11, 13, and 15 in the second frame.
In a logical system with four frames holding fourteen high nodes sharing one switch, there will only be two empty node positions (see the Frames section of Chapter 1 for clarification). In this example, the first three frames are fully populated with four high nodes in each frame. The last frame has two high nodes and two empty high node slots. This means the system has two valid switch ports associated with node slot numbers 9 and 13.
The first example is a single frame fully populated with eight wide nodes. These wide nodes occupy the odd numbered node slots. Therefore, all of the even number slots are said to be unoccupied and would have valid switch ports associated with them. These ports can be used for an SP Switch Router Adapter or |the adapter for an SP-attached server.
A second example is a single frame system with twelve thin nodes in slots 1 through 12 and a high node in slot 13. A high node occupies four slots but only uses one switch port. Therefore, the only valid switch ports in this configuration are created by the three unused node slots occupied by the high node. In other words, the switch ports are associated with node slots 14, 15, and 16.