Recovery from LED 552, 554, or 556 in AIX V4

Contents

About This Document

Causes of an LED 552, 554, or 556

Recovery Procedure

1. Turn the key to the Service position.

2. With bootable media of the same version and level as the system, boot the system. The bootable media can be ANY ONE of the following:
   • Bootable CD-ROM
   • mksysb
   • Bootable Install Tape

   Follow the prompts to the Welcome to Base OS menu.

3. Choose Start Maintenance Mode for System Recovery (Option 3). The next screen displays prompts for the Maintenance menu.
   • Choose Access a Root Volume Group (Option 1).

     At this stage the console will display information about rootvg and a menu with two options.

   • Choose Access this volume group and start a shell before mounting the filesystems (Option 2).

   If you get errors from the preceding option, do not continue with the rest of this procedure. Correct the problem causing the error. If you need assistance correcting the problem causing the error, contact one of the following:

   • local branch office
   • your point of sale
   • your AIX support center

4. Run the following commands to check and repair file systems. (The -y option gives fsck permission to repair file systems when necessary.)

      fsck -y /dev/hd1 
      fsck -y /dev/hd2 
      fsck -y /dev/hd3 
      fsck -y /dev/hd4 
      fsck -y /dev/hd9var 
   

   If any of the following conditions occur, proceed accordingly.

   • If fsck indicates that block 8 could not be read, the file system is probably unrecoverable. See step 5 for information on unrecoverable file systems.

   • If fsck indicates that block 8 could be read, but one of the following errors is given,

        "fsck:  Not an AIXV3 file system." 
        "fsck:  Not a recognized file system type." 
     

     then go to step 6.

   • If fsck indicates that a file system has an unknown log record type, or if fsck fails in the logredo process, then go to step 7.

   • If the file system checks were successful, skip to step 9.

5. The easiest way to fix an unrecoverable file system is to recreate it. This involves deleting it from the system and restoring it from a backup. Note that hd2 and hd3 can be recreated but hd4 cannot be recreated. If hd4 is unrecoverable, you must reinstall AIX. For assistance with unrecoverable file systems, contact your local branch office, point of sale, or AIX support center. Do not follow the rest of the steps in this document.

6. Attempt to repair the file system with this command:

      fsck -p /dev/hd#
   

   Replace hd# with the appropriate file system.

   Now skip to step 8.

7. A corruption of the JFS log logical volume has been detected. Use the logform command to reformat it.

      /usr/sbin/logform /dev/hd8 
   

   Answer YES when asked if you want to destroy the log.

8. Repeat step 4 for all file systems that did not successfully complete fsck the first time. If step 4 fails a second time, the file system is almost always unrecoverable. See step 5 for an explanation of the options at this point. In most cases, step 4 will be successful. If step 4 is successful, continue to step 9.

9. With the key in Normal position, run the following commands to reboot the system:

      exit 
      sync;sync;sync 
      shutdown -Fr 
   

   As you reboot in Normal mode, notice how many times LED 551 appears. If LED 551 appears twice, fsck is probably failing because of a bad fshelper file. If this is the case and you are running AFS, see step 12.

The majority of instances of LED 552, 554, and 556 will be resolved at this point. If you still have an LED 552, 554, or 556, try the following steps:

10. Repeat step 1 through step 3.

11. Run the following commands, which remove much of the system's configuration and save it to a backup directory.

       mount /dev/hd4 /mnt 
       mount /dev/hd2 /usr 
       mkdir /mnt/etc/objrepos/bak 
       cp /mnt/etc/objrepos/Cu* /mnt/etc/objrepos/bak 
       cp /etc/objrepos/Cu* /mnt/etc/objrepos 
       /etc/umount all 
       exit 
    

    Determine which disk is the boot disk with the lslv command. The boot disk will be shown in the PV1 column of the lslv output.

       lslv -m hd5 
    

    Save the clean ODM database to the boot logical volume. (# is the number of the fixed disk, determined with the previous command.)

       savebase -d /dev/hdisk# 
    

    If you are running AFS, go to step 12; otherwise, go to step 13.

12. If you are running the Andrew File System (AFS), use the following commands to find out whether you have more than one version of the v3fshelper file.

       cd /sbin/helpers 
       ls -l v3fshelper* 
    

    If you have only one version of the v3fshelper file (for example, v3fshelper), proceed to step 13.

    If there is a version of v3fshelper marked as original (for example, v3fshelper.orig), run the following commands:

       cp v3fshelper v3fshelper.afs 
       cp v3fshelper.orig v3fshelper 
    

13. WARNING: Do not proceed further if the system is a /usr client, diskless client, or dataless client.

    Recreate the boot image (hdisk# is the fixed disk determined in step 11):

       bosboot -a -d /dev/hdisk# 
    

14. If you copied files in step 12, copy the AFS file-system helper back to v3fshelper:

       cp v3fshelper.afs v3fshelper 
    

15. Turn the key to Normal position and run

       shutdown -Fr 
    

If you followed all of the preceding steps and the system still stops at an LED 552, 554, or 556 during a reboot in Normal mode, you may want to pursue further system recovery assistance from one of the following:

• local branch office
• your point of sale
• your AIX support center

For reasons of time and the integrity of your AIX operating system, the best alternative at this point may be to reinstall AIX.