All the trace files use message catalogs to display the messages, as described in PSSP: Messages Reference.
The flt file is in the /var/adm/SPlogs/css directory of each node. This file is used to log hardware error conditions about the switch, recovery actions taken by the fault service daemon, and other general operations that alter the switch configuration. The flt file is most important in the primary node where the switch initialization and servicing takes place.
Each entry in the flt file contains:
Each entry may contain this global field:
This file is an FFDC stack file, and is in the /var/adm/ffdc/stacks directory of each node. This file can be read using FFDC tools such as fcreport and fcstkrpt. For more information on FFDC services and tools, see First Failure Data Capture Programming Guide and Reference. The file contains detailed information on errors, and provides a mechanism to link errors that occur in the same flow. A link is provided between an AIX error log message to a more detailed message, or to a flow related error in the stack file.
Each error entry is provided with the debug information:
The file provides most of the information that is already included in the flt file.
The rc.switch.log file is in the /var/adm/SPlogs/css directory of each node. This file contains log information from the rc.switch script. This is the script that is responsible for starting the fault service daemon. If a node fails or is unable to start the fault service daemon, this file may contain an explanation of the failure.
The out.top file is in the /var/adm/SPlogs/css directory of each node. The fault service daemon creates a concurrent topology file. The daemon uses the expected.top file from the SDR or from the primary node's /etc/SP directory. The expected.top is created by the Eannotator command, and is an ideal network topology file. The fault service daemon combines the current status of links and nodes (devices) to the ideal file. The result is then saved to the out.top file.
The out.top file has comments added only on non-operational links or devices. An example of such a comment is:
S 15 2 tb3 1 0 E01-S17-BH-J8 to E01-N10 -4 R:
device has been removed from network - faulty
(link has been removed from network - fenced)
This means that:
The device status of the node is device has been removed from network - faulty.
SP Switch ports that have no connection are usually wrapped:
S 13 3 s 13 3 E01-S17-BH-J3 to E01-S17-BH-J3 2 L: initialized (wrap plug is installed)
The possible device and link status are listed with their relative recovery actions under SP Switch device and link error information.
The dist_topology.log file is in the /var/adm/SPlogs/css directory of the primary node. This file contains error messages that occur during the distribution of the topology file to the other nodes. The topology file distribution is done via the Ethernet and not through the SP Switch. Therefore, a failure to distribute the topology file is not directly related to the operation of the SP Switch.
The worm.trace file is in the /var/adm/SPlogs/css directory of each node. This file contains fault service daemon entries related to those in the flt file, but these entries are more detailed.
On a secondary node, the worm.trace file entries trace the initialization of the node connection to the SP Switch. On a primary node, the entries trace the whole switch initialization process: initializing each switch chip, calculating the routes to each node, establishing a primary-backup node, and setting the Time-Of-Day (TOD).
The fs_daemon_print.file file is in the /var/adm/SPlogs/css directory of each node. This file contains the commands that the fault service daemon responded to. A time stamp along with a message is logged in the file whenever an event occurs. Important entries are the ones that refer to the adapter initialization, route table calculating, and downloading the table into the adapter.
The daemon.stdout file is in the /var/adm/SPlogs/css directory of each node. This file is a redirection of the stdout and contains the fault service daemon initial information taken before the establishment of any other log mechanisms.
The logevnt.out file is in the /var/adm/SPlogs/css directory on the control workstation. The file contains records of errors which occurred in the components running on the node which experienced the error. These components are notified when switch-related error log entries are made, and report the summary data to Event Management for transmission to the control workstation. This file is then gathered into the snapshot compressed package. For more information on the snapshot, see Information to collect before contacting the IBM Support Center.
The summlog.out file is in the /var/adm/SPlogs/css directory on the control workstation. This file contains error information for the css.summlog daemon, which gathers summary log information and writes it to the summary log file. For information on the summary log file, see Summary log for SP Switch, SP Switch2, and switch adapter errors. summlog.out is a text file. This file is then gathered into the snapshot compressed package. For more information on the snapshot, see Information to collect before contacting the IBM Support Center.
The daemon.stderr file is in the /var/adm/SPlogs/css directory of each node. This file is a redirection of the stderr and contains the error output from the fault service daemon during initialization.
The dtbx.trace file is in the /var/adm/SPlogs/css directory of each node. The file contains trace information of the last run of the css0 adapter diagnostics. The adapter diagnostics can run in two modes: Power-On-Self-Test (POST), and direct invocation, by issuing the command diag. The file creation time:
Midnight Dec 31 1969
indicates that the file was created during the POST, when the time had not yet been set. For more details, see SP Switch adapter diagnostics.
The dtbx.failed.trace file is in the /var/adm/SPlogs/css directory of each node, if this file exists. This file contains trace information of the last failed run of the css0 adapter diagnostics. When the adapter diagnostics fail, they rename the dtbx.trace file to dtbx.failed.trace, in order to preserve the preliminary failure information. For more details, see SP Switch adapter diagnostics.
The cable_miswire file is in the /var/adm/SPlogs/css directory of the primary node, if it exists. This file reports any miswire detected by the fault service daemon during SP Switch initialization (running the worm subsystem).
The css.snap.log file is in the /var/adm/SPlogs/css directory of each node. This file is created every time that the css.snap command is run. This can be automatically on failure, or by direct invocation - when the user issues the css.snap command. This file contains information on what happened during the command invocation, and in particular:
The Ecommands.log file is in the /var/adm/SPlogs/css directory on the control workstation. This file contains trace information for the Ecommands.
The spd.trace file is in the /var/adm/SPlogs/css directory on the control workstation. This file contains messages about the SP Switch advanced diagnostics tests and architecture components. See SP Switch and SP Switch2 advanced diagnostic tools.