01EM320_076_045 System firmware fixes in EM320_083 System firmware changes that affect all systems DEFERRED AND HIPER A problem was fixed that, under certain rarely occurring circumstances, an application could cause a processor to go into an error state, and the system to crash. DEFERRED and HIPER The system initialization settings were changed to reduce the likelihood of a system crash under certain circumstances. HIPER A problem was fixed that caused the system to terminate abnormally with SRC B131E504. HIPER A problem was fixed that caused a system to fail to reboot after a B1xxE504 SRC was logged, due to a processor interconnection bus failure. The same SRC, B1xxE504, was logged when the reboot failed. HIPER A problem was fixed that might cause a partition to crash during a partition migration before the migration was complete. DEFERRED A problem was fixed such that under certain rare circumstances, if a service processor failover occurred, the new secondary service processor was not able to communicate with the system. A problem was fixed that caused SRC B1818A10 to be erroneously generated after the successful installation of system firmware. Enhancements were made to the firmware to improve the FRU callouts for certain types of failures of the time-of-day clock circuitry. A problem was fixed that, under certain rarely occurring circumstances, caused the system to crash if an L2 or L3 cache failure occurred. The firmware was enhanced so that the contents of /tmp are included when a service processor dump is taken. A problem was fixed that caused a predictive SRC, B181EF88, to be erroneously logged after a successful installation of system firmware, and a subsequent slow-mode IPL, of the system. A problem was fixed that, under certain rarely occurring circumstances, caused the system to crash with SRC B7005191 being logged. A problem was fixed that prevented the system from rebooting if an error occurred during a memory-preserving IPL. A problem was fixed that, under certain rare circumstances, caused a partition shutdown or partition reboot to hang with SRC D200B077. A problem was fixed that, under certain rarely occurring circumstances, caused the hypervisor to loose its communication link to the service processor and log SRC A181D000. A problem was fixed that, under certain rarely occurring circumstances, might have caused dynamic LPAR (DLPAR) operations on memory to fail. A problem was fixed that prevented I/O hardware operations from completing before dynamic LPAR (DLPAR) operations were performed on memory. This caused PCI bus errors, and multiple instances of SRC B7006971 to be logged. A problem was fixed in the hypervisor that, under certain rarely occurring circumstances, caused a system-level activation to fail. A problem was fixed that caused SRC B7006971 to be generated because the firmware was incorrectly performing operations on PCI-Express I/O adapters during dynamic LPAR (DLPAR) operations on memory. A problem was fixed that might have caused a processor checkstop after a node repair or node add operation. A problem was fixed that caused the message "BA330000malloc error!" to be displayed on the operating system console after a partition migration, even though SRC BA330000 had not been logged. When this problem occurred, the partition migration appeared to be successful. However, a process within the partition was either hung or had failed, and in most cased the partition had to be rebooted to fully recover. The firmware was enhanced to improve the description and service actions that are logged with SRC BA210012. A problem was fixed that, under certain rare circumstances, prevented a partition migration from completing successfully if processors were removed from the partition being migrated prior to the migration using dynamic LPAR (DLPAR) operations. A problem was fixed that, under certain rare circumstances, caused a system crash during partition migration operations. A problem was fixed that, under certain rare circumstances, caused the hypervisor to crash when it was booting. System firmware changes that affect certain systems On systems that are managed by a hardware management console (HMC), a problem was fixed that caused the HMC to show an "Incomplete" state after it attempted to read a file with an incorrect size from the service processor (or system controller). This problem also occurred if the "factory configuration" option was used on the advanced system management interface (ASMI) menus. On systems with I/O drawers attached, a problem was fixed that might have caused some I/O slots in the drawers not to be configured when the system was booted. On systems running AIX, a problem was fixed that prevented the diagnostic commands in AIX (diag and lsmcode, for example) from working after a partition migration. On i5 partitions using IOP-based I/O adapters which are configured to use i5 clustering (SAN), a problem was fixed that caused the failover of an I/O drawer or tower, to a system which previously owned the drawer or tower, to fail. On systems with a large number of fibre channel disks, a problem was fixed that caused SRC BA210003 to logged (which called out the fibre channel adapter) when the system management services (SMS) boot firmware was searching for a boot disk. In systems with clustered processors, various problems were fixed in the InfiniBand interconnection networks. 01EM320_046_031 9117-MMA systems shipped beginning in May of 2008 have a unique system VPD value which identifies them to the HMC. These systems will ship with EM320_059 or higher system firmware installed. Installing a level of system firmware lower than EM320_059 will result in an error condition with SRC B155A46B being logged when the system is powered on. New features and functions Support for the concurrent addition of a node (drawer). Support for the "cold" repair of a node (repair with power off while other nodes are running) was added. Support for IPv6. Support for logical volumes bigger than 2 TB was added. On systems managed by an HMC, virtual switch support for virtual Ethernet devices was added. This requires the HMC to be running at level HMC V7 R3.3.3.0 with efix MH01102. System firmware changes that affect all systems DEFERRED and HIPER The processor initialization settings were changed to reduce the likelihood of a processor going into an error state and causing a checkstop or system crash. HIPER A problem was fixed in the hypervisor that might cause a partition migration to fail. HIPER A problem was fixed that caused large numbers of enhanced error handling (EEH) errors to be logged against the 4-port gigabit Ethernet adapter, F/C 5740, under certain circumstances. HIPER On systems with a redundant service processor installed and enabled, a problem was fixed that caused a communications hang between the two service processors. When this occurred, it triggered a reset/reload of the primary service processor and the resulting fail-over to the secondary service processor failed in such a way that the system crashed. The firmware was enhanced to improve the system memory error recovery. A problem was fixed that caused the /tmp directory on the service processor to fill up, which results in an out-of-memory condition. When this problem occurred, the service processor usually performed a reset/reload. This is one possible cause of SRC B1817201 being logged. A problem was fixed that caused panel function 02 to fail when trying to set the "next IPL speed" or "next IPL side". The firmware was enhanced so that serial port S1 is not automatically designated the local console, even if the console is not selected within 60 seconds when the system is first booted. This enhancement allows the console to be selected again, if no selection was made on previous boot, instead of defaulting to the S1 port. HIPER A problem was fixed that caused capacity-on-demand (COD) data to be retrieved in an unreadable format from the Anchor (VPD) card. A problem was fixed that caused enhanced error handling (EEH) to fail on certain I/O adapters. DEFERRED: A problem was fixed that caused the system to appear to hang with C10090B8 in the control (operator) panel during a slow mode boot. A problem was fixed that prevented the processor clock from being deconfigured with the fabric bus after a hardware error. A problem was fixed that caused the L2 deconfiguration option to be displayed on the advanced system management interface (ASMI) menus on systems on which it is not supported. A problem was fixed that caused the GX adapter slot reservation option to be displayed on the advanced system management interface (ASMI) menus on systems on which it is not supported. Fixes problem where wrong slot location was provided in message when no slot reservations were available for adding next Feature Code 1800 or 1802 adapter. A problem was fixed that caused the location code reported with enhanced error handling (EEH) errors on certain imbedded slots have a "-Cx" suffix instead of the correct "-T#" suffix for the underlying adapter. This also impacted the HMC's System Planning tool. A problem was fixed that caused the Linux boot loader to lose its command line parameters (and fail to boot a Linux partition) during a reconfiguration reboot. A problem was fixed that caused the "iSCSI" and "network1" aliases to be created incorrectly in the SMS menus; this might have prevented the system or partition from booting from that device. A problem was fixed that caused this informational message to be erroneously sent to the operating system console: subq[5][0] destination address is 0!!! Check whether the subq is needed. If it is, allocate MEM. A problem was fixed that caused the AIX command, lsvpd, to hang if it was executed during a partition migration. A problem was fixed that caused the system or partition to hang at the "Welcome to AIX" banner, following an iSCSI boot, during the installation of AIX. A problem was fixed that caused an iSCSI login to fail under certain circumstances. When this failure occurred, the message sent to the console looked something like this: iscsiFailed to LOGIN to target, rc = 1 failed to login. could not open target 0x9034751 :system04 for r/w, aborting... tcpOPEN: iscsi open failed !BA012010 ! A problem was fixed that caused the location codes of devices attached to the integrated USB ports to have a duplicate port suffix. For example, when this problem occurred, the location code of the device was shown as: /usb-scsi@1 U789D.001.DQDGARW-P1-T2-T2-L1 instead of the correct location code, which is /usb-scsi@1 U789D.001.DQDGARW-P1-T2-L1 Two translation issues were fixed. The first one caused the string "No alias" to always be displayed on the iSCSI menus in SMS in English even though it should have been translated into the other languages that the SMS menus support.. The second one caused the NIC (network interface card) parameters such as the client IP address in the SMS ping menu to be displayed with message strings in English; these should have been translated as well. A problem was fixed that caused the SMS menus to drop into the open firmware prompt with the message "DEFAULT CATCH!" when the ping test failed. A problem was fixed that prevented the operating system from setting the boot device list in NVRAM. A problem was fixed that caused approximately 20-25 occurrences of informational SRC B7005300 to be logged during every IPL, which was filling up the error logs. On systems managed by an HMC, a problem was fixed that prevented the "100 Mbps/full duplex" setting for the HEA 1 Gbps ports from being implemented from the HMC. When this occurred, there was no error message on the HMC, but the setting never took effect. HIPER A problem was fixed that might cause the system to terminate while IPLing partitions soon after a system boot. On systems managed by an HMC, a problem was fixed that caused the MAC addresses displayed on the HMC, in the HEA logical port information for the second port group, to show invalid addresses. A problem was fixed that caused a service processor dump to be generated with SRC B181EF88 when the advanced system management interface (ASMI) client was closed abruptly, or a network failure disconnected the client and the ASMI. Enhancements were made to improve the field replaceable unit (FRU) isolation for phase-locked loop (PLL) clock failures on multi-CEC drawer system. SRCs B114F6D2, B114F6C1, B113F6C1, B157F12E, B18187EF, and B158E500 were typically seen with this type of failure. Enhancements were made to the error analysis firmware to provide better FRU callouts for certain types of processor fabric bus failures. SRCs B114E504, B114B2DF, and B181B10B were typically seen with this type of failure. A change was made to the firmware such that predictive SRCs B18138B0, B1813862, or B1813882 are now logged as informational. System firmware changes that affect certain model MMA systems: On system using the EnergyScale(TM) technology, enhancements were made to include status, log, and error information about the Power Save mode in the service processor error logs. On systems with redundant service processors enabled, a problem was fixed that caused the "restore factory configuration" function on the Advanced System Management Interface (ASMI) to fail. On systems with 7134-G20 drawers attached, a problem was fixed that caused the InfiniBand I/O device to drop packets, which resulted in an unrecoverable error. On systems with 7134-G20 drawers attached, a problem was fixed that caused the drawer to fail when performing concurrent maintenance on the associated InfiniBand loop. On systems with 7134-G20 drawers attached, a problem was fixed that caused the partition to become unresponsive when an InfiniBand cable in a redundantly-cabled loop was disconnected. 01EM320_040_031 HIPER A problem was fixed that caused a concurrent firmware installation to hang with SRC BA00E840 being logged. This problem may also cause a partition migration to hang, under certain circumstances, with the same SRC, BA00E840, being logged. This SRC will be logged when this level of firmware is installed and will generate a call home; it should be ignored. It will not be logged during subsequent installations. HIPER On systems with redundant service processors installed and enabled, a problem was fixed that caused the system to crash if a service processor failover occurred when the VPD files were being synchronized. HIPER On systems with redundant service processors installed and enabled, the firmware was enhanced so that if a failure occurs during a service processor failover, the firmware will attempt to reset/reload one of the service processors. This may allow the system to recover and stay up instead of crashing. HIPER A problem was fixed that caused the firmware to erroneously log VPD errors against the processors. This prevented drawers from powering on. 01EM320_031_031 Fixes that affect all model MMA systems DEFERRED A problem was fixed that caused a system crash (with SRC B131E504) by changing the initialization settings of the I/O control hardware. A problem was fixed that could cause the hypervisor to hang after a reset/reload of the service processor. A problem was fixed that, under certain circumstances, caused the InfiniBand adapter to stop responding to InfiniBand requests. A problem was fixed that caused SRC B1813014 to be logged after a successful system firmware installation. This SRC will be logged when this level of firmware is installed and will generate a call home; it should be ignored. It will not be logged during subsequent installations. The FRU list was changed so that clock card failures in a multi-drawer system will be easier to debug and require fewer parts to fix. A problem was fixed that caused the service processor to get stuck in a reset/reload loop, which prevented the system from booting to standby. System firmware changes that affect certain model MMA systems: On systems with redundant service processors enabled, a problem was fixed that could cause a significant increase in system boot time. On systems with two service processors installed and with redundancy disabled, a problem was fixed that caused the secondary service processor to go into the dump state, and remain in the dump state, after a platform dump. On systems with redundant service processors, SRCs B1813833 and B1813834, which were being logged intermittently after a side-switch IPL, were changed to informational. On systems with a 1519-100 tower attached, a problem was fixed that caused the location code of a connector on the integrated virtual IOP to be displayed as Un-SE1-SE1-T1 instead of Un-SE1-T1. On systems with 7134-G30 I/O drawers attached in certain cabling configurations, a problem was fixed that prevented the I/O port labels from being displayed for the port location codes on the hardware topology screens. On systems with a large number of I/O towers attached, a problem was fixed that caused the HMC to go to the incomplete state when an additional tower was added to a loop. On systems running system firmware level EM320_031, a problem was fixed that caused attempts to call home to fail. On systems running the Active Energy Manager(tm), a problem was fixed that caused the Exhaust Heat Index (EHI), as shown on Active Energy Manager(tm), to read up to 21 degrees lower than the actual exhaust air temperature. 01EM320_001_001 New features added Support for redundant service processors with failover on model MMA systems. Support for the Integrated Virtualization Manager on model MMA systems. Support for the concurrent addition of a RIO/HSL adapter on model MMA systems. Support for the concurrent replacement of a RIO/HSL adapter on model MMA systems. Support for the creation of multiple virtual shared processor pools (VSPPs) within the one physical pool. (In order for AIX performance tools to report the correct information on systems configured with multiple shared processor pools, a minimum of AIX 5.3 TL07 or AIX 6.1 must be running.) Support for the capability to move a running AIX or Linux partition from one system to another compatible system with a minimum of disruption. Support for the collection of extended I/O device information (independent of the presence of an operating system) when a system is first connected to an HMC and is still in the manufacturing default state. Improved VPD collection time on model MMA systems. Support for EnergyScale(tm) and Active Energy Manager(tm). For more information on the energy management features now available, please see the EnergyScale(tm) white paper. System firmware changes that affect all 9117-MMA systems HIPER A problem was fixed that caused a time-out in a hardware device driver. This time-out must include both SRCs B181B920 and B181D147. Other SRCs may be present including, but not limited to, B1xxB9xx, B1xxE504, and B150D141. Occassionally, the system crashes. If B181B290 and B181D147 SRCs are logged, check for any resources that were deconfigured at the time of these errors and reconfigure them using the ASMI menus. No hardware should be replaced. To recover from this error condition, the service processor must be reset by removing, then reapplying, the managed system's power. On multi-drawer model MMA systems, a problem found in testing was fixed which, when the L3 cache was disabled, under very unique (and rare) circumstances may result in data being overwritten in the cache and the system to crash. Although the exposure to this issue is very low, and there have been no reported problems from the field, the system impact if this occurred would be high. Product Engineering recommends that you schedule time to install this deferred fix at your earliest convenience.