IBM Books

Installation and Migration Guide


Deleting a frame, node, SP-attached server, or clustered enterprise server

When you delete a frame, node, SP-attached server, or clustered enterprise server, from your system, you should first plan how the change will affect the remainder of your system. Consider the workload and applications currently running on the hardware to be deleted and plan how to transfer the workload and applications to equivalent SP resources.

Note:
When you delete a node, the attached SP expansion I/O units are also deleted. When you delete a frame, the I/O units attached to nodes on that frame are also deleted regardless if they physically reside on that same frame or on a different frame.

Step 1: Archive the SDR

Before reconfiguring your system, you should back up the SDR by issuing:

SDRArchive

Note the location and the name of the file created after you issue this command.

Step 2: Unpartition your system (SP Switch only)

If your existing system has multiple partitions defined and you want to delete a frame, you need to bring the system down to one partition before you can delete the additional frame.

Note:
The remaining system partition must contain all of the security settings from all of the system partitions before you can power off any nodes for security reasons.

Step 2.1: Repartition your system to a single system partition (SP Switch only)

See the "Managing system partitions" chapter in the PSSP: Administration Guide for instructions on partitioning your SP system.

Step 3: Reconfigure LoadLeveler to remove the frame data from the LoadLeveler cluster

For more information on this step, see the "Administration tasks for parallel jobs" chapter in IBM LoadLeveler for AIX 5L: Using and Administering.

|Step 4: Reconfigure IBM Virtual Shared Disk to remove the frame data from the IBM Virtual Shared Disk cluster

|

|For more information on this step, see PSSP: Managing Shared |Disks.

Step 5: Shut down SP-attached servers, clustered enterprise servers, or the nodes in the frame

Use the cshutdown command to shut down the nodes, SP-attached servers, or clustered enterprise servers that you are deleting from your system. For example, to shut down node number 48 and nodes 16-31 in your system, enter:

cshutdown -G -N 48 16-31

Step 6: Disconnect the node from the frame (optional)

Perform this step only if you are deleting nodes from a frame that will continue to be part of the SP system. Your IBM Customer Engineer (CE) performs this step. See RS/6000 SP: Installation and Relocation for instructions.

|To delete an existing LPAR from a pSeries 690 server attached to |your SP or clustered enterprise server system, you will need to use the HMC |interface to remove the logical partition from the server. Perform the |following steps:

  1. |Start the HMC interface. This can be done in one of the following |ways: |
  2. |Select the pSeries 690 system object and follow HMC procedures for |deleting a logical partition. Refer to the Hardware Management |Console for pSeries Operations Guide for details on performing this |operation.

Step 7: Unconfigure DCE-related information for the node (required for DCE)

After the node has been deleted from the frame, you must remove any DCE-related principles and objects from the DCE registry. You must also unconfigure DCE (Admin portion).

On the control workstation, use the |rm_spsec -t admin node_dce_hostname command first, then do a DCE Admin unconfigure for the node (smit rmdce).

Notes:

  1. You must have cell administrator authority to perform this step.

  2. To remove any additional principals related to the node using the SMIT panels, enter the host name of the adapter to be deleted. For example, on the "Admin only unconfiguration for another machine" panel in the "Machine's name or TCP/IP address" field, enter the host name for the additional adapters.

  3. |For the nodes being removed, verify that all DCE principals have |been deleted from the DCE registry. Issue:
    |dcecp -c principal catalog -simplename

  4. To run this command off of the SP, you must set the SP_NAME environment variable on the remote workstation to point to the SDR of the SP system being reconfigured. Refer to the rm_spsec command in PSSP: Command and Technical Reference for a description of the -r (remote) flag.

Step 8: Disconnect SP expansion I/O units from the frame (optional)

If you are deleting a node that has attached SP expansion I/O units, also disconnect all I/O units from the frames in which they reside. Your IBM Customer Engineer (CE) performs this step. See RS/6000 SP: Installation and Relocation for instructions.

Step 9: Disconnect the hardware to be deleted

Your IBM Customer Engineer (CE) performs this step.

Step 10: Delete information from the SDR

Perform one of the following steps depending on what you are deleting.

Step 10.1: Frame, SP-attached server, or clustered enterprise server information

Perform this step if you are deleting a frame, SP-attached server, or clustered enterprise server from your system. The following example deletes the last two frames and all of the nodes contained in those frames:

spdelfram 3 2

|Step 10.2: Clustered enterprise servers migration servers

| | |

|Perform these steps if you are deleting all of your SP frames to convert |your SP-attached servers to clustered enterprise servers.

|Step 10.2.1: Delete all SP-attached server switch adapters
| |

|Perform this step if you are deleting all of your SP frames from a switched |SP system to convert your SP-attached servers to clustered enterprise |servers. Delete all switch adapter information from the SDR for your |SP-attached servers before deleting your SP frames in Step 10.2.2: Delete all SP frames.

|spdeladap 5 1 0 css0

|Step 10.2.2: Delete all SP frames
| |

|All of the SP frames in your system must be deleted in a single |operation. For example, issue:

|spdelfram -c -l 1,4

Step 10.3: Node information

Perform this step only if you are deleting a node from a frame that will continue to be part of your configuration. Use the spdelnode command to delete the node information from the SDR. For example, to delete node 17, issue:

spdelnode 2 1 1

Step 11: Set up system partitions (SP Switch only)

If you want to partition your system, you can select an alternate configuration from a predefined set of system partitions to implement before booting the nodes or you can use the System Partitioning Aid to generate and save a new layout. Follow the procedure described in the "Managing system partitions" chapter in the PSSP: Administration Guide and refer to information in "The System Partitioning Aid" section of the "Planning SP system partitions" chapter in the RS/6000 SP: Planning, Volume 2, Control Workstation and Software Environment. You do not have to partition your system now as part of this reconfiguration. You can partition it later.

Step 12. Refresh authorization files in the system or in a system partition

Remote commands depend on authorization files on the control workstation and on the nodes for root access. When a node is deleted, these authorization files must be updated to remove the entry for the deleted node on all other nodes and on the control workstation that had knowledge of the deleted node. This could be all nodes in a system partition, all nodes in system partitions with the same authentication method, or all nodes in the entire SP system.

To update the authorization files, issue:

dsh -w node_list updauthfiles

|Step 13: Refresh RSCT subsystems

To refresh the subsystems, issue the following command on the control workstation:

syspar_ctrl -r -G

This command refreshes all the subsystems in each partition such as hats so that it no longer recognizes the deleted hardware.

Step 14: Additional switch configuration

Frames with Switches

If you have deleted a frame with a switch, you will need to perform Step 14.1: Select a topology file through Step 14.4: Storing the switch topology file in the SDR and Step 14.6: Start the switch. If you have an SP Switch, you will need to perform Step 14.1: Select a topology file through Step 14.6: Start the switch.

Nodes or SP-attached servers (SP Switch only)

If you have only deleted nodes or an SP-attached server, you will need to perform only Step 14.3: Annotating a switch topology file and Step 14.4: Storing the switch topology file in the SDR.

Step 14.1: Select a topology file

Select the correct switch topology file by counting the number of node switch boards (NSBs) and intermediate switch boards (ISBs) in your system, then apply these numbers to the naming convention. The switch topology files are in the /etc/SP directory on the control workstation.

|NSBs are switches mounted in slot 17 of frames containing nodes or |SP Switch2 switches mounted in slots 2 through 16 of frames designed as |multiple NSB frames. Multiple NSBs are used in systems that require a |large number of switch connections for SP-attached servers or clustered |enterprise server configurations. ISBs are switches mounted in the |switch frame. ISBs are used in large systems, where more than four |switch boards exist, to connect many processor frames together. |SP-attached servers never contain a node switch board, therefore, never |include non-SP frames when determining your topology files.

The topology file naming convention is as follows:

expected.top.NSBnumnsb.ISBnumisb.type

where:

For example, expected.top.2nsb.0isb is a file for a two frame and two switch system with no ISB switches.

The exception to this naming convention is the topology file for the SP Switch-8 configuration, which is expected.top.1nsb_8.0isb.1.

See the Etopology command in PSSP: Command and Technical Reference for additional information on topology file names.

Step 14.2: Managing the switch topology files

The switch topology file must be stored in the SDR. The switch initialization code uses the topology file stored in the SDR when starting the switch (Estart). When the switch topology file is selected for your system's switch configuration, it must be annotated with Eannotator, then stored in the SDR with Etopology. The switch topology file stored in the SDR can be overridden by having an expected.top file in /etc/SP on the primary node. Estart always checks for an expected.top file in /etc/SP before using the one stored in the SDR. The expected.top file is used when debugging or servicing the switch.

Notes:

  1. Be aware that Estart distributes the topology file to all the nodes in the system partition on the switch. In the case of expected.top, this is significant because if the topology file is left on a node and the primary is changed to that node, the topology file will be used. If you have an expected.top file in /etc/SP on any of the nodes, make sure that you remove it when it is no longer needed.

  2. Depending upon your configuration, the first Estart of the switch may take longer than subsequent Estarts.

  3. |In a two plane SP Switch2 system, the function of |/etc/SP/expected.top is taken over by |/etc/SP/expected.top.p0 for plane 0 and by |/etc/SP/expected.top.p1 for plane 1.

Step 14.3: Annotating a switch topology file

Use the Eannotator command to update the switch topology file's connection labels with their correct physical locations. Use the -O yes flag to store the switch topology file in the SDR. Using Eannotator makes the switch hardware easier to debug because the switch diagnostics information is based on physical locations.

For example, to annotate a two-switch or maximum 32-node system, enter:

Eannotator -F /etc/SP/expected.top.2nsb.0isb.0 \
           -f /etc/SP/expected.top.annotated -O yes

Step 14.4: Storing the switch topology file in the SDR

If you entered Eannotator -O yes or yes on the Topology File Annotator menu in Step 14.3: Annotating a switch topology file, skip this step.

Use the Etopology command to store the switch topology file in the SDR and make sure that it has been annotated. For example, to store a two-switch or maximum 32-node configuration, enter:

Etopology /etc/SP/expected.top.2nsb.0isb.0.annotated

Step 14.5: Set the switch clock source for all switches (SP Switch only)

Use SMIT or the Eclock command to initialize the switch's clock source. The SMIT and Eclock interfaces require that you know the number of Node Switch Boards (NSBs) and Intermediate Switch Boards (ISBs) in your RS/6000 SP system.

Select the Eclock topology file from the control workstation's /etc/SP subdirectory, based on these numbers. For example, if your RS/6000 SP system has six node switch boards and four intermediate switch boards, you would select /etc/SP/Eclock.top.6nsb.4isb.0 as an Eclock topology file.

See PSSP: Command and Technical Reference for the Eclock topology file names.

Use the Eclock command to set the switch's clock source for all switches.

For example, if your RS/6000 SP system has six node switch boards and four intermediate switch boards, select /etc/SP/Eclock.top.6nsb.4isb.0 as an Eclock topology file. Enter:

Eclock -f /etc/SP/Eclock.top.6nsb.4isb.0

This command sets the proper clock source settings on all switches within a 96-way (6 nsb, 4 isb) RS/6000 SP system.

To verify the switch configuration information, enter:

splstdata -s

Step 14.6: Start the switch

Issue the Estart command to start the switch.

Note:
|If you are using the Switch Admin daemon for node recovery, start it |by issuing startsrc -s swtadmd on SP Switch systems or startsrc |-s swtadmd2 on SP Switch2 systems before issuing the Estart |command. |


[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]