File Name: Linux_RDAC_readme.TXT Software package for IBM TotalStorage™ DS4000 Linux RDAC multipath driver. Version : 09.10.A5.01 Last Update Date: 11/22/2004 Refer to the IBM TotalStorage™ Disk Storage Systems Technical Support web site for the latest information and updated readme file. Products Supported: ---------------------------------------------------------------- | New Model | Old Model | Machine Type | Model | |------------|-----------|--------------|------------------------| | DS4500 | FAStT 900 | 1742 | 90X, 90U | |------------|-----------|--------------|------------------------| | DS4400 | FAStT 700 | 1742 | 1RX, 1RU | |------------|-----------|--------------|------------------------| | DS4300 | FAStT 600 | 1722 | 60U, 60X (Turbo Option)| ---------------------------------------------------------------- CONTENTS --------- 1.0 Overview 2.0 Change History 3.0 Installation and Setup Instructions 4.0 Configuration Information 5.0 Unattended Mode 6.0 Web Sites and Support Phone Number 7.0 Trademarks and Notices 8.0 Disclaimer 1.0 Overview: 1.1 Description The Linux RDAC provides multipath failover/failback support for the logical drives in the DS4000 storage server that are mapped to the Linux host server. The Linux host server must have Fibre Channel (FC) connections to the host ports of both controllers A and B of the DS4000 storage server. It is provided as a multipath driver alternative to the Linux FC host bus adapter failover device driver. Please refer to the Dependencies section for the required code levels and Linux kernel versions. In addition, please refer to the http://www.ibm.com/servers/storage/support/disk/ IBM TotalStorage™ Disk Storage Systems Technical Support web site for the latest supported DS4000 Storage server models, host adapters, device drivers, Linux kernel versions and updated readme. Note: Deviations from the listed code levels are not supported. Please contact IBM Support or representatives for help if required. 1.2 Limitations 1. The Linux RDAC driver cannot co-exist with a HBA-level multipath failover/failback driver such as the 6.06.63-fo driver. You might have to modify the driver make file for it to be compiled in the non-failover mode. 2. SteelEye's™ LifeKeeper® Clustering is currently supported with 8 luns only. Please use the following instructions setting up the LifeKeeper® cluster. a. Recompile the Fibre Channel Host Bus Adapter driver using the following setting. - Change the setting QLA2X_PERFORMANCE to 0 in the qla_settings.h file. b. Change the following setting in the /etc/default/LifeKeeper file. LCMHBEATTIME=5 LCMNUMHBEATS=16 c. Reboot the Linux host after completing steps a and b. 3. All limitations which apply to the current DS4000 Storage Server when configured with Linux servers also apply to this Linux RDAC release. 4. DS4000 Storage Server In-band management is currently not supported. The DS4000 Storage Server Universal Access LUN (UTM) should not be mapped to the storage partition. 5. Auto Logical Drive Transfer(ADT/AVT) mode is not supported at this time. ADT(AVT) is automatically enabled in the Linux storage partitioning host type. It has to be disabled using the script that is bundled in this Linux RDAC web package or in the \Scripts directory of this DS4000 Storage Manager version 9 support for Linux CD. The name of the script file is DisableAVT_Linux.scr. Use the following steps to disable the ADT/AVT mode in your Linux host type partition. Open the DS4000 Storage Manager Enterprise Management window, - Highlight the subsystem - Select Tools - Select Execute script. - A script editing window opens. In this window - Select File - Select Load Script - Give the full path name for the script file (eg . /scripts/DisableAVT_Linux.scr) and click OK - Select Tools - Select Verify and Execute Note: This is assumed that the "Linux" host type is selected for the Storage Partitioning partition that the Linux host server HBA port definded in. This "Linux" host type has ADT/AVT enabled as default. Please refer to the IBM TotalStorage DS4000 Storage Manager version 9 Installation and Support Guide for appropriate OS or the Enterprise Window Online help for more information on how to execute a script. 7. The Linux SCSI layer does not support skipped (sparse) LUNs. If the mapped LUNs are not contiguous, the Linux kernel will not scan the rest of the LUNs. Therefore, the LUNs after the skipped LUN will not be available to the host server. The LUNs should always be mapped using consecutive LUN numbers. 8. The SMdevices program does not report LUNs beyond the LUN number that the DS4000 Storage Server Universal Access LUN (UTM LUN) is mapped to. The UTM LUN should not be mapped anyway because in-band management is currently not supported. 9. Fibre channel configuration. If a host server has multiple HBA ports and each HBA port sees both controllers (via a un-zoned switch), the Linux RDAC driver may return I/O errors during controller failover. The recommendation is to use multiple unconnected FC switches to zone the FC switch into multiple zones so that each HBA port sees only one controller in a DS4000 Storage Server. 10. The RDAC driver reports I/O failures immediately after all paths are failed. When the RDAC driver detects that all paths to a DS4000 Storage Server are failed, it will report I/O failure immediately. This behavior is different from the IBM FC HBA failover driver. The IBM FC HBA failover driver will wait for a certain time period before reporting an I/O failure to the host application. There is no work-around. 11. The virtual HBA driver can not be loaded if there no storage array attached. The work-around is to connect the host server to the DS4000 storage server and run "hot_add" utility. Note: If no LUN is mapped to the host server partition on the DS4000 storage server, hot_add will not bring up the mpp_Vhba module. 12. The cluster support feature in the Linux RDAC driver is only available for controller 05.4x.xx.xx or later firmware. If a SCSI-2 reserve/release command is addressed to a LUN on an array running a firmware version lower than 05.40, a check- condition 05-02-00 (key-asc-ascq) (illegal request, invalid command operation code) will be returned. 1.3 Enhancements N/A 1.4 Level of Recommendations and Prerequisites for the Update: N/A 1.5 Dependencies: The following code levels must be used. Deviations from the listed code levels are not supported. a. DS4000 Storage Server Controller firmware and NVSRAM: 1. Storage Controller Firmware versions: a. DS4500: FW_06101100_06100300 (06.1x.xx.xx code thread) b. DS4400: FW_06101100_06100300 (06.1x.xx.xx code thread) c. DS4300 Turbo: SNAP_06101100 (06.1x.xx.xx code thread) 2. Storage Controller NVSRAM versions - (Without FC/SATA intermix premium feature option): a. DS4500: N1742F900R910V03 (06.1x.xx.xx code thread) b. DS4400: N1742F700R910V03 (06.1x.xx.xx code thread) c. DS4300 Turbo: N1722F600R910V03 (06.1x.xx.xx code thread) 3. Storage Controller NVSRAM versions - (With FC/SATA intermix premium feature option): a. DS4500: I1742F900R910V04 b. DS4400: I1742F700R910V04 c. DS4300 Turbo: I1722F600R910V05 b. IBM DS4000/FAStT, FC-2 and FC2-133 Fibre Channel Host Bus Adapters - 7.01.01 non-failover Note: The FC HBA device driver must be non-failover version. c. Linux Kernel version - For RedHat AS 2.1 (32-Bit) : 2.4.9-e.43 up smp - For RedHat AS 2.1 (64-Bit) : 2.4.9-e.43 up smp - For RedHat AS 3.0 (32-Bit) : 2.4.21-15.0.3.EL up smp - For RedHat AS 3.0 (64-bit) : 2.4.21-15.0.3.EL up smp - For RedHat AS 3.0 AMD-64 : 2.4.21-15.0.3.EL up smp - For RedHat AS 3.0 (32-Bit) : 2.4.21-20.EL up smp - For RedHat AS 3.0 (64-bit) : 2.4.21-20.EL up smp - For RedHat AS 3.0 AMD-64 : 2.4.21-20.EL up smp - For SuSe SLES 8 AMD-64 : 2.4.21-231 up smp - For SuSe SLES 8 (32-bit) : 2.4.21-231 up smp - For SuSe SLES 8 (64-bit) : 2.4.21-231 up smp d. FAStT-MSJ 2.0.29 build 39 e. IBM DS4000 Storage Manager Host software - For 32bit Intel architecture (IA-32) servers or servers running 32bit Intel architecture version of the Linux OS kernel 1. IBM DS4000 Storage Manager Runtime version: 09.10.A5.03-1 SMruntime-LINUX-09.10.A5.03-1.i586.rpm 2. IBM DS4000 Storage Manager Client version: 09.10.G5.05-1 SMclient-LINUX-09.10.G5.02-1.noarch.rpm 3. IBM DS4000 Storage Manager Utilities version:09.10.A5.01-1 SMutil-LINUX-09.10.A5.01-1.i386.rpm 4. IBM DS4000 Storage Manager Agent version: 09.10.A5.01-1 SMagent-LINUX-09.10.A5.01-1.i386.rpm - For 64bit Intel architecture (IA-64) servers 1. IBM DS4000 Storage Manager Runtime version: 09.10.A5.03-1 SMruntime-LINUX-09.10.A5.03-1.ia64.rpm 2. IBM DS4000 Storage Manager Client version: 09.10.G5.05-1 SMclient-LINUX-09.10.G5.02-1.noarch.rpm 3. IBM DS4000 Storage Manager Utilities version:09.10.A5.01-1 SMutil-LINUX-09.10.A5.01-1.ia64.rpm 4. IBM DS4000 Storage Manager Agent version: 09.10.A5.01-1 SMagent-LINUX-09.10.A5.01-1.ia64.rpm Refer to the http://www.ibm.com/servers/storage/support/disk/ IBM TotalStorage™ Disk Storage Systems Technical Support web site for the latest update to the supported host adapters, device driver levels, Linux kernel versions, DS4000 controller firmware versions and readme. 2.0 Change History -------------------- Version 09.00.A5.01 - 11/24/2004 - Update the readme with currently supported Linux OS kernel and DS4000 Storage Manger version 9 host software and DS4000 controller firmware package version Version 09.00.A5.00 - 08/31/2004 - Support for DS4000 storage manager version 9.1 Version 08.40.A5.00 - 04/05/2004 - This is the initial release. 3.0 Installation and Setup Instructions ----------------------------------------- 3.1 Prerequisites 1. The Linux RDAC driver is released as a source-code package in the format of a gunzip compressed tar file. To unpack the driver source package, type the following command in the Linux console window: tar -zxvf ibmrdac-LINUX-xx.xx.xx.xx-source.tar.gz where xx.xx.xx.xx is the release version of the RDAC driver. The source files will uncompress to the "linuxrdac" directory. 2. The Host server must have the non-failover Fibre Channel HBA device driver properly built and installed before the Linux RDAC driver installation. Refer to the FC HBA device driver README or the FC HBA User Guide for instructions on installing the non-failover version of the device driver. The driver source tree is included in the package if you need to build it from the source tree. 3. Although the Host server can have different FC HBAs from multiple vendors or different FC HBA models from the same vendors, only one model of FC HBAs can be connected to the IBM DS4000 Storage Servers. 4. Check for the following entries in the appropriate file for a given Linux OS distribution. a. On RedHat AS 2.1 and ES 3.0 servers, it is assumed that there is a scsi_hostadapter entry for the physical HBA driver(s in the /etc/modules.conf file. For example, if the system has the IBM FC2-133 FC HBA, there should be a scsi_hostadapter entry such as alias scsi_hostadapterx qla2300 a. On SuSe SLES 8 servers, it is assumed that the low-level HBA driver names are included in the INITRD_MODULES string in the /etc/sysconfig/kernel file. For example, if the system has the IBM FC2-133 FC HBA, INITRD_MODULES string should include the word "qla2300". 5. The kernel source tree for the Linux kernel version to be built against should already be installed before RDAC driver package installation. 3.2 Build and Installation 3.2.1. SUSE Distribution Setup Instructions. For RedHat AS 2.1 or ES 3.0, proceed to step b. a. Install the kernel-source from the SUSE distribution. b. create a soft link to the kernel source. ln -sf /usr/src/ /usr/src/linux c. To ensure kernel version synchronization between the driver and running kernel, type the following in the Linux console window. Press Enter after each command. - cd /usr/src/linux - make mrproper (completely clean the kernel tree) - make menuconfig - make cloneconfig or cp /boot/vmlinuz.config .config (copy the new config) - make oldconfig (update configuration using .config) - make dep (rebuild the dependencies) - make modules (build the modules-not require on newer kernel version) 3.2.2. RDAC Driver Building for SuSe and RH. a. Change to the "linuxrdac" directory. b. To remove the old driver modules in that directory, type the following and press Enter: make clean c. To compile all driver modules and utilities in a multiple CPUs server (SMP kernel), type the following and press Enter: make 3.2.3. RDAC Driver Installation a. To copy driver modules to the kernel module tree and build the new RAMdisk image (mpp.img) which includes RDAC driver modules and all driver modules that are needed during boot time. Type the following and press Enter: make install b. Follow the instructions display at the end of the build process to add a new boot menu option which uses /boot/mpp.img as the initial RAMdisk image. IMPORTANT: If your system is a RedHat AS 2.1 or ES 3.0, please note edit the /sbin/mkinitrd script in the following area to change 3000 to 15000: if ['uname -m' = "ia64"]; then IMAGESIZE=400 else IMAGESIZE=15000 fi The following instructions may work in certain RedHat kernel version - add "ramdisk_size=15000" as a kernel boot parameter in the new boot option, similar to the following grub.conf example (Notes that it may vary in accordance with different system setup configuration): title Red Hat Linux Advanced Server-up (2.4.9-e.27smp) root (hd0,6) kernel /vmlinuz-2.4.9-e.27smp ro root=/dev/hda7 ramdisk_size=15000 initrd /mpp.img 3.3 RDAC Driver Post-Installation a. Reboot the system using the new boot menu option. b. To verify that sd_mod, sg, mpp_Upper, qla2300, and mpp_Vhba were loaded after the reboot, type the following and press Enter: /sbin/lsmod c. To verify that the RDAC driver discovered the available physical Luns and created virtual Luns for them, type the following and press Enter: ls -lR /proc/mpp d. You can now issue I/Os to the Luns. e. If you make any changes to MPP configuration file (/etc/mpp.conf) or persistent binding file (/var/mpp/devicemapping), please run mppUpdate to re-build RAMdisk image to include the new file so that the new configuration file (or persistent binding file) can be used on the next system reboot. f. To dynamically re-load the driver stack (scsi_mod, sd_mod, sg, mpp_Upper, , mpp_Vhba) without rebooting the system, you need to first comment out all "scsi_hostadapter" entries in /etc/module.conf. Then, issue "modprobe -r mpp_Upper" to unload the driver stack and follow by "modprobe mpp_Upper" to re-load the driver stack. g. The file /etc/syslog.conf should have entries for kern.debug, kern.notice, kern.info and kern.warning in order to capture all debug and error messages for MPP driver. 3.4 Disable AVT Open the DS4000 Storage Manager Enterprise Management window, - Highlight the subsystem - Select Tools - Select Execute script. - A script editing window opens. In this window - Select File - Select Load Script - Give the full path name for the script file (eg . /usr/scripts/DisableAVT_Linux.scr) and click OK - Select Tools - Select Verify and Execute Please refer to the IBM TotalStorage DS4000 Storage Manager version 9 Installation and Support Guide for appropriate OS or the Enterprise Window Online help for more information on how to execute a script. 3.5 Persistent Naming Linux SCSI device names have the possibility of changing when the host system reboots. We recommend using a utility such as devlabel to create user-defined device names that will map devices based on a unique identifier, called a UUID. As of this writing, devlabel is available as part of the Red Hat Enterprise Linux 3 distribution, or online at: http://www.lerhaupt.com/devlabel/devlabel.html. 3.6 RDAC Driver Package Uninstall and Reinstall/Update To un-install the RDAC driver package, type the following and press Enter: make uninstall Note: To Reinstall/Update the RDAC driver package, you need to issue the command "make uninstall" followed by the "make install" command under the /linuxrdac directory in the new RDAC driver source code bundle. 3.7 Kernel Update After applying the kernel update and booting into the new kernel, please do the following things to build MPP Initrd image for the new kernel: a. Change directory to Linuxrdac source code directory. b. do "make uninstall". c. Follow driver building and installation procedure described in Section 3.2.2 and Section 3.2.3. 3.9 Initial Ramdisk image (initrd) Initial RamDisk Image (initrd image) is automatically created when the driver is installed using "make install". Boot loader Configuration file needs to have entry for this newly created image. The initrd image is located in boot partition with file name :'mpp-`uname -r`.img. { Example : mpp-2.4.21-9.ELsmp.img } For a Driver Update, If the system already has old entry for MPP, the system administrator has to just modify the existing MPP entry accordingly in the boot loader configuration file. In most of the cases no chage is required if the kernel version is same. To create new 'initrd image' user has to run 'mppUpdate' which will overwrite old image file with the new image file. If 3rd party drivers needed to be added to initrd image, edit the /etc/sysconfig/kernel ( SUSE ) or /etc/modules.conf (RedHat) with the 3rd party driver entires. Run 'mppUpdate' to create new ramdisk image. 3.10 Adding and Deleting mapped logical drives (LUNs) Linux RDAC supports re-scan a newly mapped LUN without rebooting the server. The utility program is packed with the Linux RDAC driver. It can be invoked by either "hot_add" or "mppBusRescan" command. "hot_add" is a symbolic link to mppBusRescan. There are man pages for both commands. However, the Linux RDAC driver doesn't support LUN deletion. One has to reboot the server after deleting the mapped logical drives. 4.0 Configuration Information ------------------------------- 4.1 Configuration installation N/A 4.2 Configuration Settings N/A 4.3 Hardware Status and Information N/A 5.0 Unattended Mode ------------------------------- N/A 6.0 WEB Sites and Support Phone Number ---------------- 6.1 IBM TotalStorage™ Disk Storage Systems Technical Support web site: http://www.ibm.com/servers/storage/support/disk/ 6.2 IBM TotalStorage™ Marketing Web Site: http://www.ibm.com/servers/storage/disk 6.3 If you have any questions about this update, or problem applying the update go to the following HelpCenter World Telephone Numbers URL: http://www.ibm.com/planetwide IMPORTANT: You should download the latest version of the DS4000 Storage Manager host software, the DS4000 storage server controller firmware, the DS4000 drive expansion enclosure ESM firmware and the drive firmware at the time of the initial installation and when product updates become available. To be notified of important product updates, you must first register at the IBM Support and Download Web site: www-1.ibm.com/servers/storage/disk/ 7.0 Trademarks and Notices ---------------------------- The following terms are trademarks of the IBM Corporation in the United States or other countries or both: IBM FAStT TotalStorage™ the e-business logo xSeries HelpCenter Linux is a registered trademark of Linus Torvalds. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United states, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. STEELEYE are TRADEMARKS and LIFEKEEPER is a registered trademark of STEELEYE TECHNOLOGY, INC. 8.0 Disclaimer ---------------------------- 8.1 THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. IBM DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND MERCHANTABILITY WITH RESPECT TO THE INFORMATION IN THIS DOCUMENT. BY FURNISHING THIS DOCUMENT, IBM GRANTS NO LICENSES TO ANY PATENTS OR COPYRIGHTS. 8.2 Note to Government Users Include the following note after the disclaimer paragraph. Note to U.S. Government Users -- Documentation related to restricted rights -- Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corporation.