Logical Volume Manager Limitation Warnings

• In the design of Logical Volume Manager (LVM), each logical partition maps to one physical partition (PP). And, each physical partition maps to a number of disk sectors. The design of LVM limits the number of physical partitions that LVM can track per disk to 1016. In most cases, not all the 1016 tracking partitions are used by a disk. The default size of each physical partition during a mkvg command is 4MB, which implies that individual disks up to 4GB can be included into a volume group.

  If a disk larger than 4GB is added to a volume group (based on usage of the default 4MB size for the physical partition), the disk addition fails. The warning message provided will be:

  The physical partition size of <number A> 
  requires the creation
  of <number B> partitions for hdiskX. The system limitation is
  1016 physical partitions per disk. Specify a larger physical
  partition size in order to create a volume group on this disk.

  There are two instances where this limitation will be enforced:

1. The user tries to use mkvg to create a volume group and the number of physical partitions on a disk in the volume group exceeds 1016.

   The workaround to this limitation is to select from the physical partition size ranges of:

   1, 2, (4), 8, 16, 32, 64, 128, 256

   Megabytes and use the mkvg -s option.

2. The disk that violates the 1016 limitation attempts to join a pre-existing volume group with the extendvg command. The user can recreate the volume group with a larger partition size allowing the new disk to work or create a standalone volume group consisting of a larger physical size for the new disk.

   If the install code detects that the rootvg drive is larger than 4GB, it will change the mkvg -s value until the entire disk capacity can be mapped to the available 1016 tracks. This install change also implies that all other disks added to rootvg, regardless of size, will also be defined at that physical partition size.

   For RAID systems, the /dev/hdiskX name used by LVM in AIX may really consist of many non-4GB disks. In this case, the 1016 requirement still exists. LVM is unaware of the size of the individual disks that may really make up /dev/hdiskX. LVM bases the 1016 limitation on the AIX recognized size of /dev/hdiskX, and not the real physical disks that make up /dev/hdiskX.

   Limitations:

   1016 VGSA Regulations	The 1016 VGSA is used to track the staleness of mirrors. The staleness of mirrors indicates that one copy of the data will not look like the other two copies. If you are in volation of 1016, you may get a false report of a non-mirrored logical volume being stale, or you may get a false indication that one of your mirror copies has gone stale. Also, the migratepv command may fail because migratepv briefly uses mirroring to move a logical volume from one disk to another. If the target logical partition is incorrectly considered stale, the migratepv cannot remove the source logical partition, and the command will fail in the middle of migration. The reorgvg is another command that performs its actions by using temporary mirroring.
   Mirroring or migratepv	If you do not use mirroring or the migratepv command, your data is still safe as the day before you found out about 1016 violations. The data may be lost only if you are mirroring a logical volume, and: All copies go bad at the same time, and LVM is not aware of it because copies that go bad are beyond the 1016 tracking range. In this case, you would still lose data if you were within the 1016 range. If you do not use mirror or the migratepv command, this issue would not be a problem.
   Move Volume Group	A volume group can be moved between systems and versions of AIX. The enforcement of this 1016 limit is only during mkvg and extendvg. The data is safe on all versions of AIX.
   Change the PP Size Limitation	The PP size limitation will not be changed. The ability to change the PP size assures that regardless of the size of your disk drive, you will be able to accommodate the drive below 1016 limit. Also, if a change to the limitation is made, then a huge incompatibility would occur in the LVM.
   Rebuild the Volume Group	The later versions of AIX mentioned in this document only prevent the future creation of disks in volume groups that will violate the 1016 limitation. Disks that already violate the 1016 limit must be recreated at larger PP sizes.

• In some instances, the user will experience a problem adding a new disk to an existing volume group or in creating of a new volume group. The warning message provided by LVM is:

  Not enough descriptor area space left in this 
  volume group.
  Either try adding a smaller PV or use another volume group.

  On every disk in a volume group, there exists an area called the volume group descriptor area (VGDA). This space allows the user to take a volume group to another AIX system and importvg the volume group into the AIX system. The VGDA contains the names of disks that make up the volume group, their physical sizes, partition mapping, logical volumes that exist in the volume group, and other pertinent LVM management information.

  When the user creates a volume group, the mkvg command defaults to allowing the new volume group to have a maximum of 32 disks in a volume group. However, as bigger disks have become more prevalent, this 32-disk limit is usually not achieved because the space in the VGDA is used up faster, as it accounts for the capacity on the bigger disks. This maximum VGDA space, for 32 disks, is a fixed size which is part of the LVM design. Large disks require more management-mapping space in the VGDA, causing the number and size of available disks to be added to the existing volume group to shrink. When a disk is added to a volume group, not only does the new disk get a copy of the updated VGDA, but all existing drives in the volume group must be able to accept the new, updated VGDA.

  The exception to this description of the maximum VGDA is rootvg. To provide AIX users more free disk space, when rootvg is created, mkvg does not use the maximum limit of 32 disks that is allowed into a volume group. Instead in AIX 3.2, the number of disks picked in the install menu of AIX is used as the reference number by mkvg -d during the creation of rootvg. Beginning with AIX 4.1, this -d number is 7 for one disk and one more for each additional disk picked. For example, if two disks are picked, the number is 8 and if three disks are picked, the number is 9 , and so on. This limit does not prohibit the user from adding more disks to rootvg during post-install. The amount of free space left in a VGDA, and the number size of the disks added to a volume group, depends on the size and number of disks already defined for a volume group.

  If the customer requires more VGDA space in the rootvg, then they should use the mksysb, and migratepv commands to reconstruct and reorganize their rootvg (the only way to change the -d limitation is recreation of a volume group).

  Note: It is recommended that users do not place user data onto rootvg disks. This separation provides an extra degree of system integrity.

• The logical volume control block (LVCB) is the first 512 bytes of a logical volume. This area holds important information such as the creation date of the logical volume, information about mirrored copies, and possible mount points in the journaled filesystem (JFS). Certain LVM commands are required to update the LVCB, as part of the algorithms in LVM. The old LVCB is read and analyzed to see if it is a valid. If the information is valid LVCB information, the LVCB is updated. If the information is not valid, the LVCB update is not performed and the user is given the warning message:

  Warning, cannot write lv control block 
  data.

  Most of the time, this is a result of database programs accessing raw logical volumes (and bypassing the JFS) as storage media. When this occurs, the information for the database is literally written over the LVCB. Although this may seem fatal, it is not the case. Once the LVCB is overwritten, the user can still:
  • Expand a logical volume
  • Create mirrored copies of the logical volume
  • Remove the logical volume
  • Create a journaled filesystem to mount the logical volume
  There are limitations to deleting LVCBs. The logical volumes with deleted LVCB's face possible, incomplete importation into other AIX systems. During an importvg, the LVM command will scan the LVCB's of all defined logical volumes in a volume group for information concerning the logical volumes. If the LVCB is deleted, the imported volume group will still define the logical volume to the new AIX system, which, is accessing this volume group, and the user can still access the raw logical volume. However, any journaled filesystem information is lost and the associated mount point won't be imported into the new AIX system. The user must create new mount points and the availability of previous data stored in the filesystem is not assured. Also, During this import of logical volume with an erased LVCB, some non-jfs information concerning the logical volume, which is displayed by the lslv command, cannot be found. When this occurs, the system uses default logical volume information to populate the logical volume's ODM information. Thus, some output from lslv will be inconsistent with the real logical volume. If any logical volume copies still exist on the original disks, the information will not be correctly reflected in the ODM database. The user should use the rmlvcopy and mklvcopy commands to rebuild any logical volume copies and synchronize the ODM.