The SP System uses the standard logs provided by both AIX and the public domain software it includes, as well as SP-specific logs. Some logs reside on the control workstation only, and some reside only on the SP nodes. Others reside on both. Table 4 summarizes and shows the location of the logs to use when diagnosing SP problems. The abbreviation CWS stands for control workstation in this table.
Your IBM Support Center representative may ask you to provide information
from these logs.
| Type of message | Log file name | Location |
|---|---|---|
| Output of the phoenix.snap tool. See phoenix.snap dump. | /tmp/phoenix.Snap/all.timestamp.tar.Z | CWS, nodes |
| Output of the SDR_test command when run without root authority | /tmp/SDR_test.log | CWS, nodes |
| Standard AIX error log entries including the SP Switch | /var/adm/ras/errlog | Nodes |
| Error messages and verbose messages from security programs. | /var/adm/SPlogs/auth_install/log | CWS, nodes |
| Automounter messages | /var/adm/SPlogs/auto/auto.log | CWS, nodes |
| Messages from the cstartup command | /var/adm/SPlogs/cs/cstart.timestamp.pid | CWS |
| Messages from the cshutdown command | /var/adm/SPlogs/cs/cshut.timestamp.pid | CWS |
| Details of recovery actions for the IBM RVSD function | /var/adm/SPlogs/csd/vsd.debuglog | Nodes |
| Summary of recovery actions for the IBM RVSD function | /var/adm/SPlogs/csd/vsd.log | Nodes |
| Trace output of pssp_script, which performs the post install customization | /var/adm/SPlogs/css/$nim_client_shr.config.log | Nodes |
| Switch cable miswire information | /var/adm/SPlogs/css/cable_miswire | Primary node |
| Messages from the css.snap script | /var/adm/SPlogs/css/css.snap.log | CWS, nodes |
| Switch admin daemon messages | /var/adm/SPlogs/css/cssadm.debug | CWS |
| Switch admin daemon messages (stdout) | /var/adm/SPlogs/css/cssadm.stdout | CWS |
| Switch admin daemon messages (stderr) | /var/adm/SPlogs/css/cssadm.stderr | CWS |
| Fault service daemon messages for the SP Switch2 | /var/adm/SPlogs/css/daemon.log | Nodes |
| Fault service daemon messages (stderr) for the SP Switch | /var/adm/SPlogs/css/daemon.stderr | Nodes |
| Fault service daemon messages (stdout) for the SP Switch | /var/adm/SPlogs/css/daemon.stdout | Nodes |
| System error messages that occurred while distributing the topology file to the nodes. | /var/adm/SPlogs/css/dist_topology.log | Primary node |
| Trace of SP Switch adapter diagnostics failures | /var/adm/SPlogs/css/dtbx_failed.trace | Nodes |
| SP Switch adapter diagnostics trace information | /var/adm/SPlogs/css/dtbx.trace | Nodes |
| Messages from adapter diagnostics (stderr) | /var/adm/SPlogs/css/dtbxworm.stderr | Nodes |
| Log from the Eclock command | /var/adm/SPlogs/css/Eclock.log | CWS |
| Log from all Ecommands issued | /var/adm/SPlogs/css/Ecommands.log | CWS |
| Log from the Emonitor command | /var/adm/SPlogs/css/Emonitor.log | CWS |
| Result of Estart commands issued by the Emonitor daemon | /var/adm/SPlogs/css/Emonitor.Estart.log | CWS |
| Trace of last Eunpartition operation | /var/adm/SPlogs/css/Eunpart.file | Primary node |
| Trace file of fault service daemon messages | /var/adm/SPlogs/css/fs_daemon_print.file | Nodes |
| Switch fault information | /var/adm/SPlogs/css/flt | Nodes |
| stdout and stderr of the Event Management Resource Monitor and Methods | /var/adm/SPlogs/css/logevnt.out | CWS |
| SP Switch and SP Switch2 advanced diagnostics Messages Daemon log | /var/adm/SPlogs/css/msdg.log | CWS |
| Description of problems that arise while switch is initializing | /var/adm/SPlogs/css/out.top | Primary node |
| Initialization messages from the SP Switch support code | /var/adm/SPlogs/css/rc.switch.log | Nodes |
| Log from switch router generation | /var/adm/SPlogs/css/router.log | Nodes |
| Output log from switch router generation when it detects a failure | /var/adm/SPlogs/css/router_failed.log | Nodes |
| SP Switch advanced diagnostics tests and architecture components log | /var/adm/SPlogs/css/spd.trace | CWS, nodes |
| SP Switch2 advanced diagnostics tests and architecture components log | /var/adm/SPlogs/cssX/pY/spd.trace, where X and Y are variable. See SP Switch2 log and temporary file hierarchy. | CWS, nodes |
| SP Switch advanced diagnostics GUI log | /var/adm/SPlogs/css/spd_gui.log | CWS |
| Summary records for events logged to AIX error log on nodes. | /var/adm/SPlogs/css/summlog | CWS |
| stdout and stderr for the CSS logging daemon's Event Management client | /var/adm/SPlogs/css/summlog.out | CWS |
| Current state of the switch network, details about attached nodes and the topology file | /var/adm/SPlogs/css/topology.data | Primary node |
| Worm trace file from switch initialization | /var/adm/SPlogs/css/worm.trace | Primary node |
| SP Switch2 log files. See SP Switch2 log and temporary file hierarchy. | /var/adm/SPlogs/css0/* | Primary node |
| SP Switch2 adapter diagnostics messages. | /var/adm/SPlogs/css0/colad.trace See SP Switch2 log and temporary file hierarchy. | nodes |
| Trace information for the cssadm2 daemon. | /var/adm/SPlogs/css0/cssadm2.debug See SP Switch2 log and temporary file hierarchy. | nodes |
| Unexpected error messages received by the cssadm2 daemon, while performing commands external to the daemon. | /var/adm/SPlogs/css0/cssadm2.stderr See SP Switch2 log and temporary file hierarchy. | nodes |
| Unexpected informational messages received by the cssadm2 daemon, while performing commands external to the daemon. | /var/adm/SPlogs/css0/cssadm2.stdout See SP Switch2 log and temporary file hierarchy. | nodes |
| Errors logged by the la_event_d daemon for the SP Switch2 PCI Adapter. | /var/adm/SPlogs/css0/la_error.log See SP Switch2 log and temporary file hierarchy. | nodes |
| la_event_d daemon verbose messages and raw dump data for the SP Switch2 PCI Adapter. | /var/adm/SPlogs/css0/la_event_d.trace See SP Switch2 log and temporary file hierarchy. | nodes |
| SP Switch2 log files. See SP Switch2 log and temporary file hierarchy. | /var/adm/SPlogs/css1/* | Primary node |
| Results of the last CSS verification test | /var/adm/SPlogs/CSS_test.log | CWS |
| Output of the supper command | /var/adm/SPlogs/filec/supdate.time | Nodes |
| Actions supper performs when updating file collections | /var/adm/SPlogs/filec/supdate.timer | Nodes |
| Messages generated by the last get_keyfiles command issued | /var/adm/SPlogs/get_keyfiles/get_keyfiles.log | Nodes |
| Kerberos V4 authentication database administration daemon | /var/adm/SPlogs/kerberos/admin_server.syslog | Primary authentication server |
| Kerberos V4 primary authentication server log | /var/adm/SPlogs/kerberos/kerberos.log | Primary authentication server |
| Kerberos V4 secondary authentication server log | /var/adm/SPlogs/kerberos/kerberos.slave_log | Secondary authentication server |
| Kerberos V4 authentication database propagation daemon | /var/adm/SPlogs/kerberos/kpropd.log | Secondary authentication server |
| Messages generated by transfer of srvtab files to nodes | /var/adm/SPlogs/kfserver/kfserver.log.PID | CWS |
| Messages generated by registration process for the kfserver program | /var/adm/SPlogs/kfserver/regserver.log | CWS |
| Messages generated by the Problem Management daemon | /var/adm/SPlogs/pman/pmand.log | Nodes |
| Messages generated by the Problem Management daemon | /var/adm/SPlogs/pman/pmand.partition name.log | CWS |
| System Data Repository configuration messages | /var/adm/SPlogs/sdr/SDR_config.log | CWS |
| Output of the SDR_test command when run with root authority | /var/adm/SPlogs/SDR_test.log | CWS, nodes |
| System Data Repository error messages | /var/adm/SPlogs/sdr/sdrdlog.partition.pid | CWS |
| Login control messages | /var/adm/SPlogs/spacs/spacs.log | Nodes |
| SP configuration Vital Product Data directory | /var/adm/SPlogs/SPconfig/* | CWS, nodes |
| Vital Product Data output for the node | /var/adm/SPlogs/SPconfig/node number.umcl | CWS, nodes |
| lscfg -v command output | /var/adm/SPlogs/SPconfig/node number.lscfg | CWS, nodes |
| Messages generated by system daemons, including hardware errors | /var/adm/SPlogs/SPdaemon.log | CWS, nodes |
| Hardware Monitor hmc daemon log | /var/adm/SPlogs/spmon/hmcd/ hmcd[ipaddress].log. julian_date | CWS |
| SP extension node messages | /var/adm/SPlogs/spmgr/spmgrd.log | CWS |
| Hardware Monitor initialization and error messages | /var/adm/SPlogs/spmon/hmlogfile. julian_date | CWS |
| Node conditioning messages | /var/adm/SPlogs/spmon/nc/nc.frame.node | CWS |
| Hardware Monitor s70d daemon error messages | /var/adm/SPlogs/spmon/s70d/s70d.frame.log.julian_date | CWS |
| Activity of the logging daemon (splogd). | /var/adm/SPlogs/spmon/splogd.debug | CWS |
| Contains the PID of the logging daemon, (splogd). | /var/adm/SPlogs/spmon/splogd/splogd.pid | CWS |
| SP Logging daemon state changes | /var/adm/SPlogs/spmon/splogd.state_changes.
timestamp
Note: this log is not shipped activated. To activate, see Configuration test 1 - Check for state logging. | CWS |
| Microcode download messages recorded when using smit (smitty supervisor command) | /var/adm/SPlogs/spmon/ucode/ucode_log.frame.node | CWS |
| Output of the spmon_ctest command | /var/adm/SPlogs/spmon_ctest.log | CWS |
| Output of the spmon_itest command | /var/adm/SPlogs/spmon_itest.log | CWS |
| Job Switch Resource Table Services information and error messages | /var/adm/SPlogs/st/st_log | Nodes |
| Sysctl server log messages | /var/adm/SPlogs/sysctl/sysctld.log | CWS, nodes |
| AIX error messages from mirroring a root volume group using SP volume group commands | /var/adm/SPlogs/sysman/mirror.out | Nodes |
| System Management configuration messages | /var/adm/SPlogs/sysman/node.config.log.pid | Nodes |
| System Management first boot configuration messages | /var/adm/SPlogs/sysman/node.configfb.log.pid | Nodes |
| System Management console messages | /var/adm/SPlogs/sysman/node.console.log | CWS, nodes |
| System Management configuration messages | /var/adm/SPlogs/sysman/spfbcheck.log | Nodes |
| AIX error messages from unmirroring a root volume group using SP volume group commands | /var/adm/SPlogs/sysman/unmirror.out | Nodes |
| Informational and error messages from the SYSMAN_test command | /var/adm/SPlogs/SYSMAN_test.log | CWS, nodes |
| Event Management activity log | /var/ha/log/em.default.partition-name | CWS, nodes |
| hags internal trace and log file | /var/ha/log/hags* | CWS, nodes |
| GS service log - summary log | /var/ha/log/hags_node_incarnation.partition.long
on PSSP nodes.
/var/ha/log/hags.partition_node_incarnation.partition.long on the PSSP control workstation. | CWS, nodes |
| hagsglsm internal trace and log file | /var/ha/log/hagsglsm* | CWS, nodes |
| Trace information for the topology services daemon | /var/ha/log/hats.dd.hhmmss.partition-name | CWS, nodes |
| Information from the topology services startup script | /var/ha/log/hats.partition-name | CWS, nodes |
| Trace information for Network Interface Modules, processes used by the Topology Services daemon to monitor network interfaces. | /var/ha/log/nim.hats.interface
name.partition name.00n for PSSP
nodes
/var/ha/log/nim.topsvcsinterface name.cluster name.00n for HACMP nodes. | CWS, nodes |
| Hardmon resource monitor messages | /var/ha/run/haem.hostname/IBM.PSSP.hmrmd/ IBM.PSSP.hmrmd_log.julian-date | CWS |
| filec_config command log, no longer in use | /var/sysman/logs/* | Nodes |
| SP SNMP Agent messages | /var/tmp/SPlogs/spmgr/spgrd.log | CWS, nodes |
Log files are cleaned up on the nodes by the cleanup.logs.nodes command. Log files are cleaned up on the control workstation by the cleanup.logs.ws command. By default, continuously growing logs are trimmed to 400 lines every night, and non-growing files are deleted after seven days. The exceptions are: