IBM pSeries PCI-X Dual Channel Ultra320 SCSI RAID Adapter README

IBM eServer

PCI-X Dual Channel Ultra320 SCSI RAID Adapter
CCIN: 5703, HV 1975

 DATE: Feb 20, 2006

Table of Contents

1.0 Microcode Change History

2.0 General Information
2.1 Installation Time
2.2 Machine's Affected
2.3 AIX Requirements
2.4 Linux Requirements

3.0 Determine the Current Microcode Level for AIX
3.1 Determine the Current Microcode Level for Linux

4.0 Downloading and Unpacking the Firmware Update Package from CORE
4.1 Downloading the RPM Format File to the Target Server
4.2 Discovery Tool Microcode CD-ROM creation and download instructions

5.0 Verifying microcode before download and notes
5.1 Microcode Download Procedure-PCI-X Dual Channel Ultra320 SCSI RAID Adapter for AIX
5.2 Microcode Download Procedure-PCI-X Dual Channel Ultra320 SCSI RAID Adapter for Linux (SLES 9 SP1, RHEL4 and above)
5.3 Microcode Download Procedure-PCI-X Dual Channel Ultra320 SCSI RAID Adapter for Linux (SLES 9 base, RHEL3 and prior)

=====================================================================================

1.0 Microcode Change History:

0509007e -Dual-Initiator (HA) mode fixes:
A) Adapter transition timing window shortened for non-SCSI RAID adapters (fix for CCIN 5702, 571A)
B) Modification to cache flushing algorithm, fix for the below.
     1) A Stand Alone SCSI RAID adapter (that is not the preferred Primary) is full of cache data that will take a long time to flush.
     2) Power on a second SCSI RAID that is the preferred Primary.
     3) When the failover happens the first SCSI RAID will not have finished flushing cache data before the second IOA starts talking to the drives that the first IOA is still flushing data to (CCIN 5703 & 571B)
-Added NACA support (all U320 SCSI Controllers/Adapters)

0509007a Fixes for SCSI RAID HA (Dual Initiator) mode. This will be the first acceptable microcode level for SCSI RAID HA.

0509006d
-Contains fix for HUS10K DASD Selection Timeouts which can cause a boot drive not to be found and the system to boot to SMS.

-Error log entry w/ PRC 14005060 can appear in pSeries SCSI RAID HA environments during random system boots.
-SCSI Target Mode fix for possible hung I/O operation
-Fix for FC 5703 and U320 Cheetah 73LP-X15 drives
-SCSI RAID rebuild fix for varying DASD vendor/product IDs
-Added support for vendor unique “c0” command used in Atape & 3590 configurations w/ HVD/LVD converter and created workaround to prevent SCSI bus hang resulting in reset condition
-Enhancement to RAID 10 read performance
-Fix to prevent the following: Data may exist in the cache marked for the same section of disk. The older data was in the process of being destaged when power was lost on the adapter while the newer data was just written. The older data has a chance to be written back to disk upon power on rather than the newer. This problem was seen sporadically in test w/ the IBM Hitachi 15K Rpm DASD.

05090066
Do not install this level. Configuration Problems w/ HUS10K DASD.

-New revision level of microcode.

-Added microcode support for SCSI RAID HA mode (device driver support in Oct 2005 release of AIX). As part of this, the sisraid location code has changed. See the below example. Changes in Red.

Location code in 0309006a and prior

sisraid0 U7311.D20.6592BFA-P1-C02-C1-T1

Location code beginning in 05090066

sisraid0 U7311.D20.6592BFA-P1-C02-T3

-Ported 20MBps SE fix to new revision level.

-Fixed FCode bug which caused BA090003 sysplanar errors to be logged during boot from a rootvg SCSI RAID array.
NOTE: Some sysplanar errors may get logged after downloading this level of microcode. Review of the timestamp will reveal that these errors are from a prior boot. Clear the errorlog ("errclear 0") after downloading the microcode to prevent future improper logging of this error.

0309006a -Added detection for invalid DMA's. This DMA issue represents a potential data error. IBM recommends that you install this fix immediately.
-Cache Data Boundary modifications

03090068
-Minimum support level for HV products
-Increased BATs speed (Adapter/Controller comes available faster on bringup)
-Added support for HV products
-Alterations to the SCSI RAID cache page tables
-Alterations to SCSI RAID cache flushing routine including increased drain speed
-Support for increased cache size from 16MB to 40MB in SCSI RAID Enablement Card

03090061
Prior to microcode update, a shutdown is performed to the I2C bus to acquiesce activity. This prevents the I2C bus from getting hung and the Virtual SES devices from coming up "defined".

0309005f
-Virtual SES code fix (SF2, L4).
1) (SF2, L4) I2C bus may hang when adapter reset occurs while the adapter is reading the EEPROM on the I2C bus. Reduce window of occurrence.
2) (L4). I2C bus timeout when adapter is reset and running diagnostics.
-SCSI RAID fix.
1) Fix for Adapter Unit Check when Fast/Write Cache not operational (e.g. bad battery, bad NVRAM stick).
2) Added Notification of battery/charger not working.
3) Enhanced error log for Read Cache error .
4) Change in cache implementation from Stack (LIFO) to Queue (FIFO).

0309005d
-Virtual SES fixes: a) LED fix for L4 system and b) PCI-X Quad Channel Ultra320 SCSI RAID Adapter fix for all systems
-Error log changes: a) Differentiated a error condition. Adapter now logs a specific error instead of generic for a given situation. b) Return the correct error information to the pSeries Adapter Device Driver (ADD) so a SCSI adapter detected CRC error is logged instead of a program error microcode problem. Problem was seen on L4, SF2 and SF4.
-Added Linux support features and fixed Linux drive replacement problem
-Force SE mode to 20MBps when an SE device is connected to the 7029-6C3/6E3 new feature backplane. This backplane expands the Integrated SCSI Controller adding an external port.

03090059
External SE device fix; external SE devices are limited to 20MBps.
Added support for unique vendor command on 3580/3583 LTO tape device.
Modification to run Ultra320 speeds using Linux kernel 2.6.
Various performance enhancements.

03090051
Resolves SCSI_ERR10 (SCSI Bus Reset) condition upon Diagnostic's System Verification or Microcode Download in 7028-6C1 w/ safte device.

0309004d
Enhanced SCSI signal path validation.
Removed extended Target Mode delays.
Removed erroneous AIX errors logged during drive microcode download.
Added unique RAID array identifiers for Linux support.
Removed erroneous SCSI bus reset error logged when a device reports an error condition.
Corrected extended SCSI bus reset that may occur when running HACMP.

03090042
Installing two 4-pack backplanes in a p615 may cause the second backplane (sesX) to boot in the defined state or not be seen. Microcode 0042 corrects this problem.

0309002f
GA level

Note: In order for Ultra320 to function properly a minimum level of microcode is required for the DASD. These are listed below

Drive Type	Microcode Level (Hex)	Microcode level (ASCII as appears in VPD)
Cheetah73LP (9GB)	C54E	43353445
Cheetah73LP (18GB+)	C50E	43353045
Cheetah146LP	C50F	43353046
Cheetah73LP-X15	C51A	43353141
Ultrastar73	S5DE	53354445
US146Z10	S28C	53323843

2.0 General information

SUBJECT: PCI-X Dual Channel Ultra320 SCSI RAID Adapter

******* PLEASE READ THIS ENTIRE NOTICE *********

2.1 Installation Time:
Approximately 30 minutes

2.2 Machine's Affected:
7028-6C1/6E1,6C4/6E4
7029-6C3/6E3
7038-6M2
7039-651
7040-671/681/61D/61R
7311-D10/D20

9110-510
9111-285/520
9112-265
9113-550

9114-275

9115-505
9117-570

9118-575

9119-590/595

9123-710
9124-720

9131-52A

9133-55A
9406-520/550/750

2.3 AIX Requirements:
For support: AIX 5.1 ML_04 or AIX 5.2 ML_01
For U320 support: AIX 5.1 ML_05, AIX 5.2 ML_02, AIX 5.3 or above along w/ the aforementioned minimum DASD microcode requirements.

2.4 Linux Requirements:
For support: SLES 8 SP3 or RHEL 3 U1 or above

Notes:
-SLES9 SP1 and RHEL4 introduce the functionality of the "iprconfig" utility which is referenced in the instructions for Linux sections, below.
-If you are using SLES9 base level, RHEL3 or below then please follow the instructions for AIX through section 5.0. To flash the microcode continue to section 5.3.
-"iprconfig", the tool referenced in the Linux instructions, may be available in a Linux RPM package. The tool set is called "iprutils".

3.0 Determine the Current Microcode Level for AIX:

Before you install the microcode, it is important to determine the microcode level of the Adapter installed in the target system. Use

the following instructions to read the ROM level stored in the Adapter's VPD.

1) List the SCSI RAID Adapters installed in the system by typing:

lsdev -C | grep sisioa

Note: All of the SCSI RAID initiators addresses that are installed will be displayed. The names will be sisioaX, where X is 0, 1, etc.

2) To check the current microcode level for the adapter or controller enter the following command

lscfg -vl sisioaX

Where X is the number/s of the previous command. The Command will produce output similar to

DEVICE LOCATION DESCRIPTION

sisioa0          U0.1-P1-I5 PCI-X Ultra320 SCSI RAID Adapter

      PCI-X Dual Channel Ultra320 SCSI RAID Adapter:
        Part Number.................97P2702
        FRU Number..................97P2702
        Serial Number...............YL1093175020
        Manufacture ID..............0009
        EC Level....................0
        ROM Level.(alterable).......0309002f
        Product Specific.(Z0).......5703
        Product Specific.(Z1).......1
        Device Specific.(YL)........U0.1-P1-I5

If the ROM Level is less than 0509007e then you should update the microcode.

3.1 Determine the Current Microcode Level for Linux:

RHEL 3:
1) To list the scsi initiators Type: "ls /proc/scsi/ipr"

The Command will produce output similar to
"0 1 2"
2) To check the current microcode level for the adapter or controller Type: "cat /proc/scsi/ipr/X" (where X is the number from the previous command).
The Command will produce output similar to
"Firmware Version: 0309002f"

SLES 8 SP4:
1) To list the scsi initiators Type: "ls /proc/scsi/ibmsis"
The Command will produce output similar to
"0 1 2"
2) To check the current microcode level for the adapter or controller Type: "cat /proc/scsi/ibmsis/X" (where X is the number from the previous command)
The Command will produce output similar to
"Firmware Version: 0309002f"

SLES 9 and RHEL 4:
1) Type "iprconfig"
2) Select "Display hardware status"
3) Select SCSI adapter by typing a "1" in front of it and pressing Enter.
4) The Command will produce output similar to
"Firmware Version . . . . . . . . . . . . : 03090061"

If the Firmware Version is less than 0509007e then you should update the microcode.

4.0 Downloading and Unpacking the Firmware Update Package from CORE

NOTE: The CORE image will now be the RPM Format File.

1) After downloading the files from CORE on to your personal system find the following files in the CORE destination directory (c:\ibmdocs\DISKIMG):
A) "pci.1069B166.09-0509007e-1.aix5.1.noarch.rpm"
B) "pci.1069B166.09-0509007e-1.Linux.noarch.rpm"
2) Log in to the Target Server
3) Create a directory, if not already existent (mkdir /tmp/microcode; mkdir /tmp/microcode/RPM)
4) Transfer the file to the Target Server "/tmp/microcode/RPM directory" using ftp or another method. Change to that directory, "cd /tmp/microcode/RPM".
5a) For AIX, SLES9 base, RHEL3 & prior: Unpack the RPM file by using the following command: "rpm -ihv --ignoreos pci.1069B166.09-0509007e-1.aix5.1.noarch.rpm"
5b) For SLES9 SP1, RHEL4 & above: Unpack the RPM file by using the following command: "rpm -ihv --ignoreos pci.1069B166.09-0509007e-1.Linux.noarch.rpm"
6a) For AIX, SLES9 base, RHEL3 & prior: The microcode files will be added to /usr/lib/microcode/.
6b) For SLES9 SP1, RHEL4 & above: The microcode files will be added to /lib/firmware/.

4.1 Downloading the RPM Format File to the Target Server
Use this method to download to an AIX or Linux system:

Note: The instructions that follow are specific AIX/Linux commands. AIX/Linux commands are CASE (lower and upper) SENSITIVE, and must be entered exactly as shown, including filenames.

1) Make a directory on an AIX/Linux system to receive the RPM format file.
Enter: "mkdir /tmp/microcode"

Enter: "mkdir /tmp/microcode/RPM"
2) Transfer the RPM format file to the /tmp/microcode/RPM directory (using "Save as ..."). Change to that directory, "cd /tmp/microcode/RPM".

You'll see the filenames for the RPM files.
    A) For AIX, SLES9 base, RHEL3 & prior, the filename is pci.1069B166.09-0509007e-1.aix5.1.noarch.rpm
    B) For SLES9 SP1, RHEL4 & above, the filename is pci.1069B166.09-0509007e-1.Linux.noarch.rpm
3) Unpack the file by executing the instructions below:
    A) For AIX, SLES9 base, RHEL3 & prior, type "rpm -ihv --ignoreos pci.1069B166.09-0509007e-1.aix5.1.noarch.rpm"
    B) For SLES9 SP1, RHEL4 & above, type "rpm -ihv --ignoreos pci.1069B166.09-0509007e-1.Linux.noarch.rpm"
4a) For AIX, SLES9 base, RHEL3 & prior: The microcode files will be added to /usr/lib/microcode/.
4b) For SLES9 SP1, RHEL4 & above: The microcode files will be added to /lib/firmware/.

4.2 Discovery Tool Microcode CD-ROM creation and download instructions

To obtain information how to burn a CD-ROM and run the Discovery Tool for an AIX System please go to: CDROMhelp.html
1) After running the Discovery Tool successfully the "/tmp/microcode/RPM" directory was created and your rpm files are copied from the CD-ROM.

2) Change to that directory, "cd /tmp/microcode/RPM".

3) Unpack the file by executing the instructions below:
Enter the commands:
rpm -ihv --ignoreos pci.1069B166.09-0509007e-1.aix5.1.noarch.rpm
4) Microcode files will be copied to "/etc/microcode". The file size and checksum of the microcode image will be verified in Section 5.0

Notes:
- "/etc/microcode" is a symbolic link to "/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory
or file then the user will be prompted for a new location.

5.0 Verifying microcode before download and notes

NOTE: Microcode download must be performed separately for each Adapter bus under distinct Logical Partitions (LPAR's).
Please save current and older versions of the microcode update files in case you need to restore your system.

NOTE: Microcode Download CAN be performed concurrently.

Installation Time: Approximately 30 min System Time.

Please verify the file size and checksum of the raw microcode files matches the below.
Filesize: 1250896
Checksum: 45913
For AIX, SLES9 base, RHEL3 & prior: "ls -l /usr/lib/microcode/5052414D.0509007e" to verify Filesize.
"sum /usr/lib/microcode/5052414D.0509007e" to verify Checksum.
For SLES9 SP1, RHEL4 & above: "ls -l /lib/firmware/pci.1069B166.09.0509007e" to verify Filesize.
"sum /lib/firmware/5052414D.0509007e" to verify Checksum.

For systems that have LPAR installed, the microcode file must be placed on all partitions that have a SCSI device in order to update the code. Follow this procedure for each partition:

For AIX, SLES9 base, RHEL3 & prior:
    A) Log into the target partition.
    B) Type: "cd /usr/lib/microcode"
    C) FTP (for AIX) or SFTP (for Linux) to the partition that has the media bay assigned. (ie. ftp/sftp partition1.ibm.com)
    D) Type: "cd /usr/lib/microcode"
    E) If using FTP then Type: "bin"
    F) Type: "get 5052414D.0509007e"
    G) Type: "quit"

For SLES9 SP1, RHEL4 & above:
    A) Log into the target partition.
    B) Type: "cd /lib/firmware"
    C) SFTP to the partition that has the media bay assigned. (ie. "sftp partition1.ibm.com")
    D) Type: "cd /lib/firmware"
    E) Type: "get pci.1069B166.09.0509007e"
    F) Type: "quit"

5.1 Microcode Download Procedure-PCI-X Dual Channel Ultra320 SCSI RAID Adapter for AIX

Procedure:
1) It is advised, but not necessary to stop all applications.
2) Enter diag
3) Select the "Task Selection" from diagnostics menu.
4) Select "Download Microcode" or "Microcode Tasks -> Download Microcode" from "Task Selection" menu.
5) Select all of the PCI-X Dual Channel Ultra320 SCSI RAID Adapters that need to be updated from the list of devices by pressing "Enter". The devices will be displayed as sisioa#, where # is the number of the device. Press "F7".
6) A dialogue box will be displayed on screen. Press "Enter" to continue.
7) Select "/usr/lib/microcode".
8) A dialogue box will may be displayed on screen. It will state that the current microcode level on the adapter is not in the /usr/lib/microcode directory. This is acceptable as you will be flashing new microcode onto the adapter. Also, incorrect code will automatically be rejected. Press "Enter" to continue.
9) An "M" will be displayed next to the latest level of microcode in the /usr/lib/microcode directory. Select that level and press "Enter".
10) The following message will appear on the screen when download is completed: "Microcode download complete successfully. The current microcode level for the device sisioa# is ..... Please run diagnostics on the adapter to ensure that it is functioning properly." DO NOT run diagnostics on the device at this time.
11) Exit diagnostics.
12) Verify the code level is 0509007e by typing "lscfg -vl sisioa#" for each scsi device updated. <Where # is the number sisioa device>

5.2 Microcode Download Procedure-PCI-X Dual Channel Ultra320 SCSI RAID Adapter for Linux (SLES9 SP1, RHEL4 & above)

Procedure:
1) Stop all applications.
2) Type "iprconfig" and press "Enter".
3) Select "Download Microcode" and press "Enter".
4) Select all of the PCI-X Dual Channel Ultra320 SCSI RAID Adapters that need to be updated from the list of devices by pressing "1". Then press "Enter".
5) Select the appropriate microcode image to flash by pressing "1". (The image will end in 0509007e). Press Enter. Press Enter again to confirm.
6) If multiple devices were selected in step 4, then repeat step 5 until finished.

5.3 Microcode Download Procedure-PCI-X Dual Channel Ultra320 SCSI RAID Adapter for Linux (SLES9 base, RHEL3 and prior)

Procedure:
1) Stop all applications.
2) From the Command Prompt: Type "iprupdate" and press "Enter".

0509007e	-Dual-Initiator (HA) mode fixes: A) Adapter transition timing window shortened for non-SCSI RAID adapters (fix for CCIN 5702, 571A) B) Modification to cache flushing algorithm, fix for the below. 1) A Stand Alone SCSI RAID adapter (that is not the preferred Primary) is full of cache data that will take a long time to flush. 2) Power on a second SCSI RAID that is the preferred Primary. 3) When the failover happens the first SCSI RAID will not have finished flushing cache data before the second IOA starts talking to the drives that the first IOA is still flushing data to (CCIN 5703 & 571B) -Added NACA support (all U320 SCSI Controllers/Adapters)
0509007a	Fixes for SCSI RAID HA (Dual Initiator) mode. This will be the first acceptable microcode level for SCSI RAID HA.
0509006d	-Contains fix for HUS10K DASD Selection Timeouts which can cause a boot drive not to be found and the system to boot to SMS. -Error log entry w/ PRC 14005060 can appear in pSeries SCSI RAID HA environments during random system boots. -SCSI Target Mode fix for possible hung I/O operation -Fix for FC 5703 and U320 Cheetah 73LP-X15 drives -SCSI RAID rebuild fix for varying DASD vendor/product IDs -Added support for vendor unique “c0” command used in Atape & 3590 configurations w/ HVD/LVD converter and created workaround to prevent SCSI bus hang resulting in reset condition -Enhancement to RAID 10 read performance -Fix to prevent the following: Data may exist in the cache marked for the same section of disk. The older data was in the process of being destaged when power was lost on the adapter while the newer data was just written. The older data has a chance to be written back to disk upon power on rather than the newer. This problem was seen sporadically in test w/ the IBM Hitachi 15K Rpm DASD.
05090066	Do not install this level. Configuration Problems w/ HUS10K DASD. -New revision level of microcode. -Added microcode support for SCSI RAID HA mode (device driver support in Oct 2005 release of AIX). As part of this, the sisraid location code has changed. See the below example. Changes in Red. Location code in 0309006a and prior sisraid0 U7311.D20.6592BFA-P1-C02-C1-T1 Location code beginning in 05090066 sisraid0 U7311.D20.6592BFA-P1-C02-T3 -Ported 20MBps SE fix to new revision level. -Fixed FCode bug which caused BA090003 sysplanar errors to be logged during boot from a rootvg SCSI RAID array. NOTE: Some sysplanar errors may get logged after downloading this level of microcode. Review of the timestamp will reveal that these errors are from a prior boot. Clear the errorlog ("errclear 0") after downloading the microcode to prevent future improper logging of this error.
0309006a	-Added detection for invalid DMA's. This DMA issue represents a potential data error. IBM recommends that you install this fix immediately. -Cache Data Boundary modifications
03090068	-Minimum support level for HV products -Increased BATs speed (Adapter/Controller comes available faster on bringup) -Added support for HV products -Alterations to the SCSI RAID cache page tables -Alterations to SCSI RAID cache flushing routine including increased drain speed -Support for increased cache size from 16MB to 40MB in SCSI RAID Enablement Card
03090061	Prior to microcode update, a shutdown is performed to the I2C bus to acquiesce activity. This prevents the I2C bus from getting hung and the Virtual SES devices from coming up "defined".
0309005f	-Virtual SES code fix (SF2, L4). 1) (SF2, L4) I2C bus may hang when adapter reset occurs while the adapter is reading the EEPROM on the I2C bus. Reduce window of occurrence. 2) (L4). I2C bus timeout when adapter is reset and running diagnostics. -SCSI RAID fix. 1) Fix for Adapter Unit Check when Fast/Write Cache not operational (e.g. bad battery, bad NVRAM stick). 2) Added Notification of battery/charger not working. 3) Enhanced error log for Read Cache error . 4) Change in cache implementation from Stack (LIFO) to Queue (FIFO).
0309005d	-Virtual SES fixes: a) LED fix for L4 system and b) PCI-X Quad Channel Ultra320 SCSI RAID Adapter fix for all systems -Error log changes: a) Differentiated a error condition. Adapter now logs a specific error instead of generic for a given situation. b) Return the correct error information to the pSeries Adapter Device Driver (ADD) so a SCSI adapter detected CRC error is logged instead of a program error microcode problem. Problem was seen on L4, SF2 and SF4. -Added Linux support features and fixed Linux drive replacement problem -Force SE mode to 20MBps when an SE device is connected to the 7029-6C3/6E3 new feature backplane. This backplane expands the Integrated SCSI Controller adding an external port.
03090059	External SE device fix; external SE devices are limited to 20MBps. Added support for unique vendor command on 3580/3583 LTO tape device. Modification to run Ultra320 speeds using Linux kernel 2.6. Various performance enhancements.
03090051	Resolves SCSI_ERR10 (SCSI Bus Reset) condition upon Diagnostic's System Verification or Microcode Download in 7028-6C1 w/ safte device.
0309004d	Enhanced SCSI signal path validation. Removed extended Target Mode delays. Removed erroneous AIX errors logged during drive microcode download. Added unique RAID array identifiers for Linux support. Removed erroneous SCSI bus reset error logged when a device reports an error condition. Corrected extended SCSI bus reset that may occur when running HACMP.
03090042	Installing two 4-pack backplanes in a p615 may cause the second backplane (sesX) to boot in the defined state or not be seen. Microcode 0042 corrects this problem.
0309002f	GA level