Mirroring the rootvg Volume Group for AIX V4

Contents

About This Document

This document specifies the supported method for mirroring the rootvg volume group to provide high availability access of the AIX operating system. Because the rootvg contains the AIX operating system, several restrictions exist for this configuration that do not exist for the non-rootvg volume group mirroring. Therefore:

1. This document should be read in its entirety. Facts presented later in this document may affect your implementation and use of rootvg mirroring.
2. This information is only valid for AIX Versions 4.1, AIX 4.2 and AIX 4.3. When referring to APARs, three numbers may be provided: one for 4.1 fixes, one for the equivalent 4.2 fix, and a possible number for the equivalent 4.3 fix. If there is no entry for the 4.3 fix, it is implied that the fix for a problem found in 4.1 or 4.2 is in the 4.3.0 release and, therefore, no APAR needs to be applied. A table at the end of this document summarizes the fixes.
3. Users should contact their local IBM Branch office to check on the supportability of this functionality on versions of AIX released prior to AIX 4.1.
4. This function is not supported on /usr client, diskless client, or dataless client systems.

This documentation supersedes previous IBM documentation on mirroring the rootvg volume group. Updates to this document may exist; refer to the document date to determine which is the latest set of instructions for this subject.

Why Mirror the rootvg Volume Group?

1. Maintaining an active mirrored copy of the rootvg volume on another disk ensures continuous operation of the AIX operating system in the event that a disk that is part of the operating system experiences failure.
2. Such maintenance provides the ability to boot more than one disk of the rootvg in the event that another boot disk has failed. In some cases, the ability to boot from alternate disks may require some user intervention.

Restrictions That Apply to This Procedure

1. APAR ix56564 (AIX 4.1 only) must be applied on the system for mksysb to correctly restore a mirrored volume group back into a system in mirrored format. If this APAR fix is not applied, then the mksysb will still restore the rootvg onto a system; however, the base mirrors of the logical volumes will not be automatically created and must be recreated by the user. To determine if this fix is installed on a machine, type the following command:

       instfix -i -k IX56564 
   

   Consult the table at the end of this document to determine if the level of AIX you are running already contains this fix.

2. If the dump device is mirrored, the user may not be able to capture the dump image from a crash, or the dump image may be corrupted. The design of LVM prevents mirrored writes of the dump device. Only one of the mirrors will receive the dump image. Depending on the boot sequence and disk availability after a crash, the dump will be in one of the following three states:
   1. not available
   2. available and not corrupted
   3. available and corrupted

   State (a) will always be a possibility. If the user prefers to prevent the risk of encountering state (c), then the user must create a non-mirrored logical volume (that is not hd6) and set the dump device to this non-mirrored logical volume. Refer to the section "Paging and Dump Devices" in this document for further details.

3. Parallel mirroring policy (the default) must be the policy used on the base logical volumes that constitute AIX (hd1, hd2, ... hd9var). This restriction is for performance reasons only.

4. Mirror strictness (the default) must be maintained on all the base logical volumes that constitute AIX (hd1, hd2, ... hd9var). If two copies of a logical volume reside on the same disk, it would defeat and invalidate the original goal of mirroring a rootvg volume group against possible disk failure.

5. The boot logical volume, hd5 and all its mirrors, must be made up of consecutive physical partitions. Most standard AIX systems have an hd5 made up of just one physical partition. But if the user ever expands hd5, the growth of hd5 and its mirrors must be made up of the next consecutive physical partitions of each mirror's location on the disk. This may be determined with the following command:

       lslv -m hd5 
   

6. hd6 must exist in rootvg and must be an active paging device. Other paging devices on other volume groups are permitted, but hd6 (and its mirror) must exist on the disks that make up rootvg. For more details see the section "Paging and Dump Devices".

7. The disks that constitute the boot drives for AIX must be disks that are supported to boot on RS/6000 platforms. Any disk that contains hd5 and its mirrors is considered a boot drive for AIX. A machine and disk configuration may be queried to determine if that combination of machine and disk supports booting from disk. Use the following command to query:

       bootinfo -B hdiskX 
   

   If the command returns a 1 then it will be bootable by AIX. Any other value indicates that this disk is not a candidate for rootvg mirroring.

8. In case of disk failure and then replacement, users should consult the AIX System Management Guide appropriate to the level of AIX they are running.

   When replacing a mirrored volume group, the steps are identical (with the exception of Restriction [14&rbrk) for rootvg and non-rootvg volume groups:

   1. Mirror copies referenced on the bad disk are removed.
   2. The "bad disk" reference is removed (via reducevg) from the volume group.
   3. Follow the procedures to re-mirror a volume group.

9. Physical and design limitations (see step 1 of the "Procedure" section) may exist that prevent the mirroring of the rootvg.

10. When the user executes the following commands:
    

    migratepv
    reorgvg
    reducevg
    

    and the logical volume that constitutes the dump device (typically hd6) OR its mirror resides on the disk, the user must first set the dump device to /dev/sysdumpnull before executing the command.

    After the command is complete, the user may reset the dump device to the original dump logical volume.

11. Some models of RS/6000 do not support the bootlist command (see the "Procedure" section). The rootvg can still be mirrored on these systems, but the possible reboot following a failure of the original boot disk will require user intervention to re-select alternate boot disks for subsequent boots.

12. The syncvg command is not available to rootvg at the time of the varyon of the rootvg volume group. If after a mirrored rootvg has booted there exist stale logical volumes in rootvg, then the user must run syncvg on rootvg to synchronize any stale logical volumes.

13. APAR IX61186 is required for bosboot to work correctly in some cases.

14. APAR IX68483 is required for booting to work correctly on 4.2 systems.

    To determine if either fix is installed on a machine, execute the following:

        instfix -i -k <apar number> 
    

    The equivalent AIX 4.2 APAR fixes are listed in the table at the end of this document.

15. In this document's table of APAR fixes for AIX 4.1, AIX 4.2, and AIX 4.3, in addition to the mandatory fixes listed in Restrictions 5 and 17, apars relevant to LVM mirroring are listed as REQUIRED rootvg mirroring fixes a user should place on their system. Please note that the most recent mirroring fix was chosen. Older mirroring fixes, by default, are included with these latest fileset fixes. These fixes are very strongly recommended for those people using any type of logical volume mirroring on their AIX systems. Please note that fix list may change as problems are discovered and fixed.

16. In AIX 4.1, the simultaneous creation of mirrored logical volumes greater than 2GB might fail and should not be attempted. This is documented in the AIX 4.1 release notes. Instead, the user should create a logical volume with the mklv command. Then, the user should create a mirror for the logical volume with the mklvcopy command. This restriction also applies to the restoration of a mirrored rootvg from a mksysb image, if the mirrored rootvg contains a logical volume greater than 2GB. To work around this, the user should set the Create Map Files option in mksysb to YES. This restriction does not apply to AIX 4.2 and AIX 4.3 systems.

Alternative

In AIX 4.2.1, two new LVM commands were introduced: mirrorvg and unmirrorvg. These two commands where introduced to simplify mirroring or unmirroring of the entire contents of a volume group. The commands detect if the entity to be mirrored or unmirrored is rootvg, and will give slightly different completion messages based on the type of volume group.

NOTE: The mirrorvg command does the equivalent of steps 2-4 in the "Procedure" section.

Paging and Dump Devices

When the paging device is mirrored, there will be extra write cycles as the mirrored image of the paged memory is written to disk. This will affect the performance to a slight degree. The user can choose not to mirror the paging device, but this would defeat the original purpose of rootvg mirroring, which is to keep a system or program active if a disk were to fail. If the paging device were not mirrored but all the other logical volumes in rootvg were mirrored, then the behavior of the system would be equal to that of a non-mirrored rootvg that experiences a paging disk fail in the middle of operation.

As discussed previously, when a system dump is occurring, the dump image is not written to disk in mirrored form. The logic behind this fact is that if the mirroring code itself were the cause of the system crash, then trusting the same code to handle the mirrored write would be pointless. Thus, mirroring a dump device is a waste of resources and is not recommended.

There is an alternative. In AIX 4.2.1, the secondary dump device was fixed. If a valid secondary dump device exists and the primary dump device cannot be reached, the secondary dump device will accept the dump information intended for the primary dump device.

In AIX 4.1 and subsequent releases, the dump device and the paging device are installed as the same initial device, hd6. However, in rootvg mirroring the desired behavior is to have paging mirrored, but not the dump device. Therefore, the user should create a non-paging dump device and NOT have that dump device mirrored.

The mirrorvg command takes this into account. If the dump devices are also the paging device, the logical volume will be mirrored. If the dump devices are NOT the paging device, that dump logical volume will not be mirrored.

Procedure

The following steps assume the user has rootvg contained on hdisk0 and is attempting to mirror the rootvg to a new disk: hdisk1.

1. Extend rootvg to hdisk1 by executing the following:

       extendvg rootvg hdisk1 
   

   The user may encounter the following error message:

       0516-050 Not enough descriptor space left in this volume group. 
       Either try adding a smaller PV or use another volume group. 
   

   In this case, the user may not add hdisk1 to rootvg for mirroring. The user may attempt to mirror rootvg's logical volumes to another disk that already exists in rootvg and meets the criteria and restrictions listed in the "Restrictions" section above. Or, the user may attempt to add a smaller disk to the rootvg. If neither option is possible, then mirroring rootvg cannot be performed on this system.

2. Disable QUORUM by executing the following:

       chvg -Qn rootvg 
   

3. Mirror the logical volumes that make up the AIX operating system by executing the following:

       mklvcopy hd1 2 hdisk1            #  /home file system 
       mklvcopy hd2 2 hdisk1            #  /usr file system 
       mklvcopy hd3 2 hdisk1            #  /tmp file system 
       mklvcopy hd4 2 hdisk1            #  / (root) file system 
       mklvcopy hd5 2 hdisk1            #  blv, boot logical volume 
       mklvcopy hd6 2 hdisk1            #  paging space 
    (If users have other paging devices, rootvg and non-rootvg, 
    it is recommended that they also mirror those logical volumes 
    in addition to hd6; refer to the section "Paging and Dump Devices".) 
       mklvcopy hd8 2 hdisk1            #  file system log 
       mklvcopy hd9var 2 hdisk1         #  /var file system 
   

   If hd5 consists of more than one logical partition, then, after mirroring hd5, the user must verify that the mirrored copy of hd5 resides on contiguous physical partitions. This can be verified with the following command:

       lslv -m hd5 
   

   If the mirrored hd5 partitions are not contiguous, the user must delete the mirror copy of hd5 (on hdisk1) and rerun the mklvcopy for hd5, using the -m option. The user should consult documentation on the usage of the -m option for mklvcopy.

4. Synchronize the newly created mirrors with the following command:

       syncvg -v rootvg 
   

5. bosboot to initialize all boot records and devices by executing the following command:

       bosboot -a -d /dev/hdisk? 
   

   hdisk? is the first hdisk listed under the PV heading after the command lslv -l hd5 has executed.

6. Initialize the boot list by executing the following:

       bootlist -m normal hdisk0 hdisk1 
   

   WARNING: Even though this command identifies the list of possible boot disks, it does not guarantee that the system will boot from the alternate disk in all cases involving failures of the first disk. In such situations, it may be necessary for the user to boot from the installation or maintenance media: select maintenance, reissue the bootlist command leaving out the failing disk, and then reboot. On some models, firmware provides a utility for selecting the boot device at boot time. This may also be used to force the system to boot from the alternate disk.

7. Shutdown and reboot the system by executing the following command:

       shutdown -Fr 
   

   This is so that the Quorum OFF functionality takes effect.

APAR Fixes

Description          UP & MP                      APAR Level 
------------------------------------------------------------- 
mksysb fix           ix56564                         4.1 
                     ------ 
                     In 4.2.0 code                   4.2 
                     ------ 
                     In 4.3.0 code                   4.3 
------------------------------------------------------------- 
bosboot failure 
                     ix61186                         4.1 
                     ------ 
                     ix62417                         4.2 
                     ------ 
                     In 4.3.0 code                   4.3 
------------------------------------------------------------- 
dump device patch 
                     not needed                      4.1 
                     ------ 
                     ix68483                         4.2 
                     ------ 
                     In 4.3.0 code                   4.3 
-------------------------------------------------------------- 
latest device driver mirroring fixes 
                     ix60521                         4.1 
                     ------ 
                     ix70884                         4.2 
                     ------ 
                     In 4.3.0 code                   4.3 
--------------------------------------------------------------- 
latest mirrorvg/unmirrorvg fixes 
                     not needed                      4.1 
                     ------ 
                     ix72058                         4.2 
                     ------ 
                     ix72550                         4.3 
---------------------------------------------------------------          

NOTE: The preceding fixes are packaged for the clvm prpq under the cumulative APAR IX58707, except for the bosboot APARs. The clvm prpq fileset ONLY exists on AIX 4.1 systems. To determine if you have the clvm prpq installed, execute the following:

    lslpp -h prpq.clvm 

If prpq.clvm is not installed on your system, you do not need to order IX58707.