Urgent and Daily Reports

SYSTEM EXPERT FULL SERVICE REPORT SECTIONS

System Expert provides over 50 possible pre-established report sections. In addition, information, logs and other output from existing customer applications are easily added as custom report sections.

System specifics

This report section provides information on system and System Expert related variables. These variables include such items as the system name, IP address, System Expert version number and customer contact information. The data provided in this report section is required for customer support and problem determination activities.

Example:

Report Type: demo Rundate: Mon Dec 1 00:01:08 EST 1997 -0500 Expert Logid: 33391 Machine Name: ibmna OS Info: AIX ibmna 1 4 000021277600 Network Name: ibmna.ibm.com Aliases: smaugrc00 smaugsa0 IP Address: 198.133.33.5 Monitor Version: vrios: 4.04.2 Installed: 97/11/28-09:34:10 Expiry: 98/11/22 Log Seq. #: 1543 Hierarchy: ibm.ibmna Company Name: IBM Corp. Address: 11400 Burnet Rd., Austin, TX Telephone: +1 800 CALL IBM DB Administrator: Joe Wizard E-Mail: dbsupport@ibmna.ibm.com

System configuration summary

This report section summarizes the system's hardware configuration, including the machine type, the system's CPU id, the amount of installed system memory, the size of the configured paging space and the type and capacity of its disk drives.

Example:

Memory: 64 MB Machine Type: IBM RISC System/6000 Swap: 204 MB Machine Model: 360|36T|365 CPU Serial # : 000021277600 Hostname : ibmna Disk drive configuration summary: hdisk0 Available 1.0 GB SCSI Disk Drive hdisk1 Available 1.0 GB SCSI Disk Drive hdisk2 Available Other SCSI Disk Drive hdisk3 Available Other SCSI Disk Drive

Disk and file system usage - URGENT

This report section provides valuable information on disk drive, volume group and file system usage. All file systems are automatically monitored and if a file system exceeds its threshold the section is marked urgent and the details of the offending file system are provided.

Example:

File System /u01 (device lv01) is 98.12% full (threshold 90%) File System /usr (device hd2) is 90.12% full (threshold 90%) Including complete file system utilization below: Volume Free Total Free Total Percent Group PP's PP's Mbytes Mbytes Used ------------------------------------------------ rootvg 381 1263 1524 5052 69.83% Drive ---------------------------------------------- hdisk0 0 248 0 992 100.00% hdisk1 0 250 0 1000 100.00% hdisk2 0 250 0 1000 100.00% hdisk3 381 515 1524 2060 26.02% File System User Disk Percent Inodes Inodes Percent Disk System Free Free Total Used Free Total Used Device Mbytes Mbytes Mbytes ----------------------------------------------------------------------- / 5.89 5.89 12.00 50.92% 2712 4096 33.79% hd4 /home 38.33 38.33 348.00 88.99% 75869 90112 15.81% hd1 /home/devl 12.02 12.02 80.00 84.97% 13816 20480 32.54% lv04 /local 42.23 42.23 392.00 89.23% 85910 100352 14.39% lv00 /pcdos 41.22 41.22 84.00 50.93% 22121 22528 1.81% lv02 /pcwin 26.95 26.95 28.00 3.75% 8173 8192 0.23% lv06 /tmp 52.94 52.94 112.00 52.73% 27272 28672 4.88% hd3 /u01 17.46 17.46 928.00 98.12% 222833 237568 6.20% lv01 /usr 103.96 103.96 1052.00 90.12% 233651 270336 13.57% hd2 /usr/OV 13.38 13.38 128.00 89.55% 28405 32768 13.31% lv05 /var 19.36 19.36 88.00 78.00% 19668 22528 12.70% hd9var ----------------------------------------------------------------------- Totals 373.74 373.74 3252.00 88.51% 740430 837632 11.60%

Hardware and software errors - URGENT

This report section provides information on the hardware and software errors detected by such operating system functions as the AIX operating system's error logger. High severity errors, such as permanent hardware errors are noted in the Urgent report. This information is valuable to the system administrator responsible for proactive maintenance of system hardware and software and the resolution of their errors.

Example:

Aug 30 01:05 MT unit 0: powerfail event reported MT unit 1: powerfail event reported 1 Channel 0x80 Sub-Channel 0x02 Device 0x0b Hard Errors - 1 Soft Errors - 0 Total I/O Operations - 35036754 Total Misc Operations - 17371930 Errors Missed - 0 type ET_HARD severity ES_WARN entity EE_DISK name iop0/pdisk001 time 809251310 Thu Aug 30 01:01:50 1997 DATA slice: 12 class: OVERRUN - lost bus arbitration while transfering data number of overruns: 1 number of IO's completed: 52 sampling rate (seconds): 30 description: This is a summary of overruns which occurred on this disk slice. The last overrun error code and description have been logged. last error code: 4400 last error description: DM got a data overrun NOTE: error code 4400 class (OVERRUN) is not uncommon. It indicates an extremely busy controller or extend bus. I/O tuning may be appropriate. ------------------------------------------- ERROR TYPE: IENT_ERR4 / ERROR ID: 3B145117 ------------------------------------------- Sequence Number : 977156 Machine Id : 000042523700 Node Id : ibm Error Class : S Error Type : UNKN Resource Name : ent0 Error Description - UNDETERMINED ERROR Probable Causes - UNDETERMINED - SOFTWARE DEVICE DRIVER Failure Causes - CHANNEL ADAPTER - COMMUNICATIONS SUBSYSTEM - SOFTWARE PROGRAM Recommended Actions - PERFORM PROBLEM DETERMINATION PROCEDURES Detail Data - RETURN CODE - 0000 0816 - STATUS CODE - 0000 0492 - PROGRAM CHECK CODE - 0000 005C - SENSE DATA - 0000 BF00 IENT_ERR4 occurred 1 time(s):. Occurrence 1 - Fri Sep 8 16:05:30 ----------------------------------------------- ERROR TYPE: DISKETTE_ERR2 / ERROR ID: 3A9C2352 ----------------------------------------------- Sequence Number : 975534 Machine Id : 000042523700 Node Id : ibm Error Class : H Error Type : UNKN Resource Name : fd0 Resource Class : diskette Resource Type : fd Location : 00-00-0D-00 Error Description - DISKETTE DEVICE FAILURE Probable Causes - CONTROLLER Recommended Actions - PERFORM PROBLEM DETERMINATION PROCEDURES Failure Causes - DISKETTE ADAPTER Detail Data - SENSE DATA - 2232 A2C6 0010 8120 0700 FF07 0001 0210 0000 012C 0000 0014 0000 001B 0000 8C22 - 0000 0000 DISKETTE_ERR2 occurred 1 time(s):. Occurrence 1 - Wed Sep 6 14:36:06 ------------------------------------------- ERROR TYPE: SCSI_ERR6 / ERROR ID: 52DB7218 ------------------------------------------- Sequence Number : 973966 Machine Id : 000042523700 Node Id : ibm Error Class : S Error Type : TEMP Resource Name : scsi0 Error Description - SOFTWARE PROGRAM ERROR Probable Causes - SOFTWARE PROGRAM Failure Causes - SOFTWARE PROGRAM Recommended Actions - IF PROBLEM PERSISTS THEN DO THE FOLLOWING - CONTACT APPROPRIATE SERVICE REPRESENTATIVE Detail Data - SENSE DATA - 0000 0000 0000 0000 0000 0064 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 - 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 - 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 - 0000 0000 0000 0000 0000 0000 SCSI_ERR6 occurred 1 time(s):. Occurrence 1 - Tue Sep 5 05:38:55 ---------------------------------------------------------------------------- Hardware/Software Error Reporting -------------------------------------------- ERROR TYPE: TTY_TTYHOG / ERROR ID: 0873CF9F -------------------------------------------- Sequence Number : 974731 Machine Id : 000042523700 Node Id : ibm Error Class : S Error Type : TEMP Resource Name : tty3 Error Description - ttyhog over-run Failure Causes - EXCESSIVE LOAD ON PROCESSOR Recommended Actions - Reduce system load. - Reduce serial port baud rate TTY_TTYHOG occurred 37 time(s):. Occurrence 1 - Tue Sep 5 12:07:30 Occurrence 2 - Tue Sep 5 12:07:40 Occurrence 3 - Tue Sep 5 12:07:41 Occurrence 4 - Tue Sep 5 12:10:35 Occurrence 5 - Tue Sep 5 12:10:44 Occurrence 33 - Tue Sep 5 12:30:43 Occurrence 34 - Tue Sep 5 12:30:44 Occurrence 35 - Tue Sep 5 12:30:49 Occurrence 36 - Tue Sep 5 12:31:21 Occurrence 37 - Tue Sep 5 12:31:30

User-account integrity and security - URGENT

The definition for user-accounts are checked for syntax errors, inconsistencies and security vulnerabilities. In addition, user-account home directories are checked for their existence and security. Login shells are also verified for their existence and that they are executable. Warnings of syntax errors and security vulnerabilities in user-account definitions are classified as urgent.

Example:

Monitor Montior Expert Expert Expert Found Limit Limit Suppress Display 12 50 100 2 10 User name lsen; no password User name nfor; password guessed User name guru; suspicious super user account (uid 0) User name bg30; no home directory (/devel/bg30) User name wow; home directory wrong group (/usr/wow:420/193) User name shutdown; duplicate user-id (root: 0) User name fsadmin; duplicate user-id (root: 0) User name makels; invalid user-id (60001) User name nobody; wrong group for nobody (60001/-2) User name oracle7; home directory group writable (/usr/oracle7)

User-environment security - URGENT

This report section provides information on security vulnerabilities in such items as 'PATH' variable settings, default 'umask' settings and program invocations in a world-writable command file. This information is valuable to the security analyst and system administrator responsible for maintaining corporate security policies and overall optimal operation of the business solution.

Example:

File: /usr/bin/whatis, user: root (root), insecure umask (000) File: /etc/profile, user: root (root), world writable directory in path (/usr/local) File: /etc/profile, user: root (root), program contains insecure paths or commands (/usr/bin) File: /etc/profile, user: root (root), world writable directory in path (/usr/local) File: /etc/profile, user: root (root), . or relative path in PATH (.) File: /etc/profile, user: root (root), program contains insecure path or commands (/usr/bin) File: /etc/profile, user: root (root), world writable program (/usr/bin/dbrload) File: /etc/rc, user: root (root), world writable program (/usr/bin/readmsg) File: /usr/spool/cron/crontabs/root, user: root (root), world writable program (/pm/bin/Chkwtmp)

Group-account integrity and security

The definition for group-accounts are checked for syntax errors and security vulnerabilities. Warnings of syntax errors and security vulnerabilities in group-account definitions are classified as urgent.

Example:

Monitor Montior Expert Expert Expert Found Limit Limit Suppress Display 5 50 100 0 5 Group name daemon: invalid userid in group entry (notes) Group name adev: invalid userid in group entry (mwt) Group name bkup; duplicate group-id (nntp:16) Group name cron; duplicate group-id (lp: 8) Group name nobody; invalid group-id for nobody (65534/-2)

File integrity and security

This report section provides information on inconsistencies with file ownership and timestamps, inaccessable files, inappropriate placed device files and FTP file permission vulnerabilities. This information is valuable to the security analyst and system administrator responsible for maintaining the integrity and the security of the business system.

Example:

Monitor Montior Expert Expert Expert Found Limit Limit Suppress Display 5 50 100 0 5 File /etc/conf/h/4ha.h; invalid uid (204) File /user/is/etc/update.log; access time in the future (Mar 5 21:00:31 2001) File /user/is/etc/update.log; modify time in the future (Mar 5 21:00:31 2001) File /u/home/ftp/dev2/tcp; critical file for ftp is world writable (666) File /usr/ftp/etc/group; critical file for ftp should not be linked to anything (link count 2)

Directory integrity and security

This report section provides information on corrupt directories and mountpoints, directory structure problems and warnings on the security and integrity of FTP directories. This information is valuable to the security analyst and system administrator responsible for maintaining the corporate security policies for the business solutions.

Example:

Monitor Montior Expert Expert Expert Found Limit Limit Suppress Display 4 50 100 0 4 Directory /user/spool/threads; can't find system mount entry - corrupt mnttab? Directory /usr/pubftp; ftp directory should not be world writable (777) Directory /usr/pubftp/bin; ftp directory should not be world writable (777) Directory /usr/pubftp/etc; ftp directory should not be world writable (777)

Mail configuration checking

This report section provides information on possible Mail Subsystem (sendmail) security problems and configuration errors. This information is valuable to the security analyst and system administrator responsible for maintaining the integrity and operation of the Mail Subsystem for the business.

Example:

/user/spool/mail/gibb is not a mailbox; owner 169, permissions: -rw-rw---- /user/spool/mail/gor is not a mailbox; owner 156, permissions: -rw-rw----

Sendmail configuration, integrity and security - URGENT

This report section provides information on Mail Subsystem (sendmail) security problems and configuration errors. This information is valuable to the security analyst and system administrator responsible for maintaining the security, integrity and operation of the Mail Subsystem.

Example:

/usr/lib/sendmail is owned by other than root, bin, sys, adm (devl) /usr/lib/sendmail.fc older than /usr/lib/sendmail.cf /usr/lib/aliases is world writable (-rw-rw-rw-) /usr/lib/aliases is owned by other than root, bin, sys, adm (postmaster) /etc/hosts is owned by other than root, bin, sys, adm (gwarden) World writable mailbox /usr/mail/burton; permissions: -rw-rw-rw- Wrong owner for mailbox /var/spool/mail/devel1; found root, expected devel1 Insecure pipeline invocation (/usr/lib/UUname) No statistics file /usr/lib/sendmail.st

Performance data analysis - URGENT

This report section provides performance statistics when various types of system resources pass their configurable thresholds. A brief warning message is also listed and will offer a suggestion where performance testing should begin. This information is valuable to the system administrator responsible for detecting performance problems and devising action plans to maintain an optimal running system.

Example:

Program 'Thrashing' may be occuring 41% of the samples show dpspo * 6 > dpf 30 occurences in 72 samples At: 09:40 40, 226 10:40 290, 846 11:00 445, 2022 13:00 34, 129 13:20 32, 108 14:20 138, 637 14:40 181, 633 15:00 245, 1025 15:20 247, 893 15:40 127, 628 16:00 278, 775 16:20 238, 591 16:40 360, 1017 17:00 273, 1221 17:20 128, 566 17:40 244, 1428 18:00 1081, 2829 18:20 1175, 5093 19:20 1092, 5804 19:40 1085, 5841 20:00 1090, 5968 20:40 1179, 6021 21:00 1074, 6196 21:20 996, 5799 21:40 1056, 6169 22:20 1056, 5894 23:00 1139, 6226 23:20 1131, 6151 23:40 1178, 6235 24:00 1147, 6230 Waiting for I/O activity 37% of the samples show %wio > 30 27 occurences in 72 samples At: 10:00 55 10:20 61 11:00 56 11:20 64 11:40 65 12:00 59 12:20 60 12:40 31 18:00 33 18:20 72 18:40 73 19:00 75 19:20 72 19:40 76 20:00 75 20:20 78 20:40 79 21:00 76 21:20 73 21:40 74 22:00 70 22:20 74 22:40 77 23:00 76 23:20 79 23:40 79 24:00 75 Unusual swapping required 37% of the samples show %swpocc > 25 27 occurences in 72 samples At: 10:00 72 10:20 76 11:00 92 11:20 94 11:40 95 12:00 96 12:20 95 12:40 43 18:00 48 18:20 82 18:40 83 19:00 87 19:20 86 19:40 88 20:00 87 20:20 87 20:40 87 21:00 86 21:20 86 21:40 87 22:00 83 22:20 83 22:40 84 23:00 86 23:20 87 23:40 87 24:00 86

Performance data analysis

This report section provides performance statistics when various types of system resources exceed their configurable thresholds. A brief warning message is also listed and will offer a suggestion where performance testing should begin. This information is valuable to the system administrator responsible for detecting performance problems and devising action plans to maintain an optimal running system.

Example:

28 samples between 09:00:00 and 18:02:00 Disk drive cd0 is heavily used 9% of the samples show %tm_act:cd0 > 50 7 occurences in 72 samples At: 10:00 71.6 10:20 75.4 11:00 75.2 11:20 94.9 11:40 94.5 12:00 94.8 12:20 94.9 Exceptionally high Disk Transfer Rate to/from hdisk2 1% of the samples show Kbps:hdisk2 > 300 1 occurence in 72 samples At: 00:20 386.5 Programs are not fitting in memory well, physical memory may be over committed 22% of the samples show 4 * fr > sr 16 occurences in 72 samples At: 01:00 11, 38 03:20 6, 20 10:00 48, 166 10:20 41, 108 11:20 24, 92 11:40 29, 96 18:20 85, 255 19:40 99, 390 20:00 98, 371 20:20 103, 360 20:40 99, 357 21:00 103, 358 23:00 103, 364 23:20 102, 343 23:40 103, 353 24:00 103, 370 Unusually high use of paging space 27% of the samples show 2 * dpspi > dpi 20 occurences in 72 samples At: 16:20 391, 710 18:00 1758, 2596 18:20 4281, 4947 18:40 4875, 5586 19:00 5277, 5926 19:20 4963, 5624 19:40 5044, 5659 20:00 5166, 5814 20:20 5501, 6075 20:40 5277, 5839 21:00 5476, 6067 21:20 5028, 5637 21:40 5376, 5958 22:00 5060, 5664 22:20 5126, 5767 22:40 5359, 5953 23:00 5506, 6064 23:20 5491, 6065 23:40 5551, 6072 24:00 5457, 6054 Unusually high use of paging space 27% of the samples show 2 * dpspo > dpo 20 occurences in 72 samples At: 10:40 290, 542 18:00 1081, 1248 18:20 1175, 1343 18:40 837, 980 19:00 1015, 1152 19:20 1092, 1266 19:40 1085, 1236 20:00 1090, 1253 20:20 1030, 1196 20:40 1179, 1315 21:00 1074, 1214 21:20 996, 1168 21:40 1056, 1223 22:00 937, 1121 22:20 1056, 1231 22:40 1023, 1166 23:00 1139, 1284 23:20 1131, 1306 23:40 1178, 1314 24:00 1147, 1294

Disk throughput, I/O transfer rate

This report section provides information on the disk transfer rates if they exceed the configurable threshold. This information is valuable to the system administrator responsible for detecting disk and controller bottlenecks in their effort to maintain optimal system performance and reliability.

Example:

Performance analysis per disk (97/08/30) Disk bandwidth warnings Sample Bad Bad Busy Time(blocks/sec) Disk Count Count % Threshold /dev/*dsk/c1d*s* 23 3 13.0% 130.0 09:00:05(294.00) 14:00:14(364.00) 23:00:05(360.00)

Backup report

System Expert can be configured to perform daily backups of all or selected file systems to a single tape. A summary of the backup performed is included in the report section illustrated below. Backup failures are reported separately in an Urgent report section.

Example:

SeqNo Backup Method Result File System Mount Point 1 backup good hd4 / 2 backup good hd1 /home 3 backup good lv01 /home/devl 4 backup good lv03 /local 5 backup good hd3 /tmp 6 backup good hd2 /usr 7 backup good hd9var /var 8 backup good lv00 /var/spool/news 9 backup good lv02 /var/devl

Checksum changes - URGENT

This report section provides information on checksum changes to selectable files. This information is very useful for identifying and analysing changes that occurred should an unexpected security breach have taken place. The information is valuable to the security analyst and system administrator reponsible for maintaining the security, integrity and optimal operation of the business system.

Example:

Checksum changes: 6 File ------------------ /etc/acct/holidays /etc/dfs/dfstab /etc/uucp/Grades /var/adm/security /var/adm/critical /var/adm/notice

NFS imports which trust setuid and setgid

Permitting NFS supplied setuid or setgid programs to run on your local machine could result in a security breach on your system. All setuid and setgid programs should therefore be monitored closely for any changes and possible security exposures. System Expert detects when setuid and setgid programs are permitted from NFS imported file systems and generates a non urgent warning message as illustrated below.

Example:

Remote Remote Local Machine Location Mountpoint fast.rd.com /c469-ah /usr/ah /c245-ah /usr/ah /c469-rest/restricted /usr/deb1/restricted /dc2.0/PSU /usr/d2.0 /dc2.0/PSU.0mpending /usr/d2.0pending /dc2.0/PSU.0mupdates /usr/d2.0updates int.rd.com /d1.1df/PF.1debf /usr/1.1debf /d1.1df/pending1.1debf /usr/pending1.1debf /d1.1ef/pending1.1ebf /usr/pending1.1ebf /dtf/PTF.1dpending /usr/1.1dpending /dtf/PTF.1pending /usr/1.1pending

Mountpoint and device file security - Urgent

This report section provides information on other-writable mountpoint directories and devices that are other-readable or other-writabale. This information is valuable to the security analyst and system administrator responsible for resolving the security vulnerabilities introduced by these access permissions.
Example:

Monitor Monitor Expert Expert Expert Found Limit Limit Suppress Display 1 1000 10000 0 1 Mount point /cdROM: device is world read/writable (/dev/dsk/c201d2s0:644)

Back to top
Symbolic links, targets missing
System Expert may find symbolic links which do not respond. These unresolved requests for information indicate the systems' inability to access the intended data. The link may no longer be needed or may have been removed maliciously or accidentally. An unresolved request for information through a symbolic link to an NFS filesystem may indicate network congestion. Any symbolic link which does not respond to the Monitor's request is classified as Urgent.
Example:

Monitor Montior Expert Expert Expert Found Limit Limit Suppress Display 1 50 100 0 1 Symlink /user/joe/src/screen/config.h -> /user/joe/src/screen/cinfig.hpux

Back to top
Large directory size changes - URGENT
This report section provides information on significant changes in directory size. This information is valuable to the system administrator responsible for planning and forecasting future disk consumption requirements. The information may also be used to assist in identifying large directories that may have been created accidentaly or maliciously.
Example:

Large directory size changes Period From: 97:08:30 00:08:57 To: 97:08:31 00:08:59 Sizes are in blocks (512 bytes) Old Size New Size % chg Directory Name 26961 41789 +55% /var/drk

Back to top
Large directory size changes
This report section provides information on significant changes in directory size. This information is valuable to the system administrator responsible for planning and forecasting future disk consumption requirements.
Example:

Large directory size changes Period From: 97:08:30 00:08:57 To: 97:08:31 00:08:59 Sizes are in 512 size blocks Old Size New Size % chg Directory Name 20426 17817 -12% /var/spool/mail 19714 22238 +12% /var/spool/uucppublic 7399 9356 +26% /var/spool/news/news 5369 7196 +34% /var/spool/news/biz 4597 3412 -25% /var/spool/news/rec 2778 3259 +17% /var/spool/news/tor

Back to top
Large file size changes
This report section provides information on significant changes in file size. This information is valuable to the system administrator responsible for planning and forecasting file system consumption and space requirements.
Example:

Large file size changes Period From: 97:08:30 00:08:57 To: 97:08:31 00:08:59 Sizes are in blocks (512 bytes) Old Size New Size Last Change File Name 10251 10369 Aug 30 23:55 /local/lib/news/history 10287 10343 Aug 30 18:55 /local/lib/news/history.o 4885 4981 Aug 30 23:18 /home/jmm/etc/mail-list/firewallsfwd ? 4264 Aug 30 12:20 /var/drk/netscape-v11N-export.rs6000-ibm-aix3.2.tar 3482 3523 Aug 30 15:33 /home/emoen/Mail/Errors 3328 3351 Aug 30 23:30 /var/tmp/snmpd.log 2626 2645 Aug 30 14:33 /home/gmb/Mail/.record ? 2442 Aug 30 12:31 /var/drk/netscape.aa 2386 2412 Aug 30 19:53 /home/problems/other 2063 2187 Aug 30 15:33 /home/emoen/Mail/CustomerSupport 2087 2167 Aug 30 13:27 /home/emoen/Mail/Monitor ? 1823 Aug 30 12:31 /var/drk/netscape.ab 1719 1746 Aug 30 12:31 /home/gmb/Mail/ibm.canada 1435 1628 Aug 30 23:55 /local/lib/news/news.stats 1540 1591 Aug 30 17:03 /home/emoen/Mail/eXpert 1350 1371 Aug 30 14:33 /home/gmb/Mail/mdis ? 1258 Aug 30 12:52 /tmp/eci-page.Z 1196 1207 Aug 30 23:55 /local/lib/news/history.pag ? 1100 Aug 30 15:23 /var/spool/uucppublic/pagenew ? 1078 Aug 31 00:09 /var/adm/wtmp 1027 1038 Aug 30 23:30 /var/tmp/xrsasmuxd.log

Back to top
Commands with exceptional system usage
This report section provides information on cammands which have used a large amount of the systems' CPU or memory capacity. This information is valuable to the system administrator, offering insight on the system workload over a daily period.
Example:

Total CPU > 20 or Total KCORE > 1000 Command #Cmds Kcore Total CPU Elapsed Mean Mean CPU Hog Name Run Minutes Minutes Minutes Size K Minutes Factor _______________________________________________________________________ maker4X. 6 301408.6 32.73 517.34 9208.08 5.46 6.33 crack 8221 12356.16 12.77 110.33 981.23 1.10 12.02 perl 7761 11814.15 12.18 109.65 970.28 0.00 11.10 oracle 1 4534.01 1.58 1.99 2865.00 1.58 79.33 ----------------------------------------------------------------------------

Back to top
Users with exceptional system usage
This report section provides information on users that have consumed a large amount of the systems' CPU and/or memory capacity. This information is valuable to the system administrator, providing insight into the identification of high resource users and applications.
Example:

Logins with exceptional Prime/Non-prime Time Usage CPU > 20 or KCORE > 500 or CONNECT > 120 Login CPU Kcore Connect Disk # of # of UID Name Minutes Minutes Minutes Blocks Proc Sess Prime Nprime Prime Nprime Prime Nprime ____________________________________________________________________________ 0 root 26 25 5680 8565 34250 39163 6576 30913 18 4 adm 0 0 6 8 39117 33197 0 0 0 7 frnxchng 1 3 719 4234 6384 38137 0 0 0 104 lpd 0 0 6 0 3396 371 0 0 0 206 frame 29 4 266212 35669 29309 53809 0 0 0 213 payroll1 0 0 100 7 967 64 0 0 0 254 wilson 3 0 830 12 31441 650 0 0 186 260 eric 0 0 9 9 1069 1064 0 0 145 401 chris 0 0 60 0 1607 0 0 0 72 402 jimbo 2 0 547 0 10299 0 0 0 957 403 davey 2 1 384 254 9692 3525 0 0 126 406 mainsite 1 0 324 2 11371 118 0 0 475 408 rstory 0 0 1 18 47 2766 0 0 0 416 jdr 0 0 40 37 1212 1156 0 0 388 21474 nobody 0 0 1 0 859 9 0 0 0

Back to top
NFS exports, system and privileges
This report section provides information on NFS exports configuration. This information is valuable to the system administrator or security analyst performing forensic analysis of changes and system tampering should an unexpected security breach have occured.
Example:

/: Root access exported to squid satin morena swordfish lock hetera /: exported read/write to all systems /temp2: exported read/write to all systems /pc: Root access exported to ibm /pcdos: exported read/write to all systems /u01/ClipArt: Root access exported to ibm /u01/ClipArt: exported read/write to all systems /u01/inst.images: Root access exported to ibm /u01/inst.images: exported read/write to all systems /usr/lpp/HPXT: Root access exported to hpx0 hpx1 hpx2 hpx3 hpx4 /home: Root access exported to ibm wa240

Back to top
Device number changes - URGENT
This report section provides information on device number changes on the system. This information is valuable to the system administrator responsible for maintaining the optimal operation of the business system.
Example:

Device number changes: 10 Found Expected File -------- ----------- -------------- 52/128 0/513234 /dev/diag/lan0 1/128 0/523926 /dev/diag/mux0 1/65664 0/513234 /dev/diag/mux1 52/1 52/2105345 /dev/ether0 52/0 52/2105344 /dev/lan0 1/128 121/2102023 /dev/mux0 1/65664 0/523926 /dev/mux1 205/4160 121/2102023 /dev/rmt/0mn 205/4096 121/2102022 /dev/rmt/0m 1/0 121/2102023 /dev/tty0p0

Back to top
File type changes - URGENT
This report section provides information on changes to the file type of a file (ie. file vs. directory). This information is valuable to the system administrator responsible for maintaining the optimal operation of the business system.
Example:

File type changes: 1 File --------------- /usr/adm/syslog

Back to top
Group-id changes - URGENT
Changes to a file's group-id could introduce operational and security related vunerabilities or be an indication of system tampering. This report section provides information on detected changes to a file's group-id. This information is valuable to the security analyst and system administrator responsible for maintaining the integrity and optimal operation of the business system.
Example:

Group-id changes: 7 Found Expected File ----- -------- --------------------------------- 0 2 /etc/trcfmt 0 2 /usr/bin/capture 0 2 /usr/lib/nls/msg/en_US 3 0 /usr/lost+found 0 2 /usr/lpp/bos/inst_root/etc/trcfmt 300 0 /usr/lpp/bosext2/dlc/lan.err 300 0 /usr/lpp/bosext2/dlc/qllc.er

Back to top
Permissions changes - URGENT
This report section provides information on changes to file permissions. The information is valuable to the security analyst and system administrator responsible for maintaining the security, integrity and optimal operation of the business system.
Example:

Permissions changes: 10 Found Expected File ----- -------- ------------------------------------ 777 770 /dev/SRC 600 644 /etc/inittab 444 440 /etc/objrepos/lvm_lock 600 210 /etc/security/.ids 777 755 /home 4555 555 /usr/bin/capture 4555 555 /usr/bin/script 644 777 /usr/lib/cfgodm.ipl 555 550 /usr/lib/lpd/plotlbe 755 775 /usr/lib/lpd

Back to top
Resolved directory and file status errors - URGENT
This report section provides information on System Expert's ability to access a previously inaccessible file or directory. This information is valuable to the security analyst and system administrator responsible for maintaining the optimal operation of the business system.
Example:

Resolved directory and file status errors: 2 File ------------------------------------------- /var/adm/RWlogs/sysm/oracle.console.log.1 /usr/adm/pacct1

Back to top
Unresolved directory and file status errors - URGENT
This report section provides information on System Expert's inability to access a previously accessible file or directory. This information is valuable to the security analyst and system administrator responsible for maintaining the optimal operation of the business system.
Example:

Unresolved directory and file status errors: 3 File Error message -------------------------------- --------------------------------------- /SYSBCKUP lstat failed, No such file or directory /bin/as lstat failed, No such file or directory /bin/cnodes lstat failed, No such file or directory

Back to top
Security/version bit changes - URGENT
This report section provides information on changes on the "trusted computing base" bit on a file or directory. This information is valuable to the security analyst and system administrator responsible for maintaining the integrity and optimal operation of the business system.
Example:

Security/version bit changes: 2 ----- --------- --------------- 0 100000000 /etc/qconfig 0 100000000 /usr/sbin/getty

Back to top
Setuid and setgid file changes - URGENT
This report section provides information on the creation, removal, change in ownership, group ownership, permissions, modification time, size and number of hardlinks for all setuid and setgid files on the system. This information is valuable to the security analyst and system administrator responsible for maintaining the integrity and optimal operation of the business systems.
Example:

Potentially serious security breaches in set[ug]id files: File date of /usr/ingres/bin/ing.u was changed from 1997-Aug-12/15:40 to 1997-Aug-30/15:40 File size of /usr/adm/ups/upslog changed from 14216135 to 14332860 Newly created set[ug]id file /usr/ingres/bin/ing.OLD Addn'l info: -rwsr-x--x 1 ingres spq 283648 1997-Aug-30 20:30 Groupid on /local/bin/dbm changed from rsa to eci

Back to top
Control file security - URGENT
This report section provides information on vulnerable network services, universal "trusted" privileges and accessibility to the system's password file via the 'tftpd' daemon. This information is valuable to the security analyst and system administrator responsible for maintaining the security, integrity and optimal operation of the business systems.
Example:

/etc/inetd.conf: fingerd enabled /etc/inetd.conf: rexd enabled /etc/inetd.conf: rexecd enabled /etc/rc: rwhod enabled /etc/rc: rexd enabled /bin/tftp: tftp can retrieve live /etc/passwd

Back to top
Files in lost+found directories - URGENT
This report section provides information on files found by 'fsck' which do not have an entry into any directory, indicating evidence of filesystem corruption. This information is valuable to the system administrator responsible for the optimal operation of the system and correction of the corrupted file system.
Example:

Permissions/Ownership etc Poss. type File name -rw-r--r-- 1 root root 42 Aug 17 1997 awk program text /lost+found/#05177

Back to top
Temporary files found
This report section provides information on the existence of selectable temporary files such as 'core' files. System Expert can be configured to automatically remove these files on a routine basis. This information is valuable to the system administrator responsible for regular cleanup and maintenance of the system.
Example:

-rw-r--r-- 1 mustard staff 28051 Aug 27 16:37 /home/mustard/core

Back to top
Event message summary - URGENT
This report section provides information on real-time events detected daily by the Prompt subsystem for the monitored machine. This information is valuable to the system administrator, providing them with a severity ranked summary of the days critical events and a list of the individual events for their review.
Example:

Number of Severity Events Report Status ======== ========= ============= 1 4 ** URGENT 2 4 ** URGENT 3 8 -- displayed 4 1 -- displayed 5 18 suppressed 9 1 suppressed Severity Program ======== ======================== Seq Date Time Message === ======== ===== ====================== 0 upost:9999 1 97/08/30 09:47 root says: Backup starts 2 97/08/30 11:44 root says: Backup finished 1 _dflimit:0002 1 97/08/30 14:25 /sh/u2 is 97% full, it was 94%, the limit is 89% 2 97/08/30 14:35 /sh/u2 is 96% full, it was 95%, the limit is 89% 3 97/08/30 14:40 /sh/u2 is 95% full, it was 96%, the limit is 89% 4 97/08/30 15:01 /sh/u2 is 98% full, it was 97%, the limit is 89% 5 97/08/30 15:04 /sh/u2 is 100% full, it was 98%, the limit is 89% 1 dns:0001 1 97/08/30 10:38 Name Server gate unavailable 2 97/08/30 10:57 Name Server gate unavailable 1 syslog_trap:0016 1 97/08/30 08:28 08:15:49 hp3 sendmail[10529]: AA10527: SYSERR: putbody: write error: Disk quota exceeded 2 97/08/30 19:08 18:59:30 hp3 sendmail[21372]: NOQUEUE: SYSERR: Cannot freeze /usr/lib/sendmail.fc: Permission denied 1 _running:0004 1 97/08/30 13:06 cron stopped running 4 97/08/30 13:07 cron started running 2 backup_chk:0002 1 97/08/30 14:18 Daily Cron Backup Failure 2 _running:0004 1 97/08/30 13:02 snmpd stopped running 2 97/08/30 13:06 syslogd stopped running 3 97/08/30 13:07 snmpd started running 4 97/08/30 13:07 syslogd started running 2 _be_root:0012 1 97/08/30 10:27 Failed su attempt by rlinton to root on ttyq8. 2 97/08/30 11:19 Failed su attempt by rcampbel to root on ttyl5. 3 97/08/30 11:57 Failed su attempt by nmandaya to root on ttyjb. 4 97/08/30 16:12 2 failed su attempts by petew to root on ttyp1 since 16:08. 2 syslog_trap:0016 1 97/08/30 23:47 23:45:33 hp3 syslogd: restart

Back to top
Event message summary
This report section provides information on lower severity real-time events detected daily by the Prompt subsystem for the monitored machine. This information is valuable to the system administrator, providing them with a severity ranked summary of the days events and a list of the individual events for their review.
Example:

Severity Program ======== ======================== Seq Date Time Message === ======== ===== ====================== 3 _dflimit:0002 1 97/08/30 07:22 /sh/dcp/b1 is 89% full, it was 82%, the limit is 86% 2 97/08/30 14:23 /sh/u1 is 95% full, it was 89%, the limit is 85% 4 syslog_trap:0016 1 97/08/30 13:14 13:07:11 ibm snmpd[15281]: NOTICE: logging started at level 0 2 97/08/30 13:14 13:07:12 ibm snmpd[15281]: NOTICE: snmpd (15281) is starting

Back to top
Reboot report - URGENT
This report section provides information on system start times, stop times and the duration the system was unavailable. This information is used to generate the "System Availability graph" which is valuable for Management review and historical trend analysis of overall system availability.
Example:

System reboot report System down System up Duration 97/08/30 13:03:00 97/08/30 13:06:37 00:03:37

Back to top
Log and key file management report
This report section provides information on System Expert's activity of pruning , aging and removal of selectable log files. This information and activity is valuable to the system administrator responsible for regular cleanup and maintenance of the system.
Example:

File Action Backup in... ----------------------------------- ------ ------------------------------------- UUCP pruned /local/lib/news/news.stats pruned /local/lib/news/news.stats.1 /etc/passwd copied /xrsa/mon/backup/passwd.1 /etc/group copied /xrsa/mon/backup/group.1 /usr/lib/uucp/Systems copied /xrsa/mon/backup/Systems.1 /usr/lib/uucp/Permissions copied /xrsa/mon/backup/Permissions.1

Back to top
UUCP usage and reliability
This report section provides information on UUCP usage and errors that have occurred. This information is valuable to the system administrator responsible for evaluating and maintaining the performance and reliability of the UUCP subsystem.
Example:

Traffic with UUCP neighbors (97/08/30 00:32:03 to 97:08:31 00:27:26) News Articles Mail UUCP Traffic (kB) Files Accptd, Dups, Que'd Recv Sent Recvd Sent Byt/S Recv Sent eci486 0 0 0 12 5 149 118 205 28 18 Totals 0 0 0 12 5 149 118 205 28 18 UUCP file transfer UUCP line usage, incoming and outgoing (97/08/30 00:32:03 to 97/08/31 00:27:26) Tx Time Rx Time Total Port (min) (min) (min) %Util ttydp3 9.1 13.1 22.2 1.54 UUCP remote execution, incoming and outgoing (97/08/30 00:32:03 to 97/08/31 00:27:26) rnews Rcvd Sent eci486 0 4 Totals 0 4 UUCP reliability (97/08/30 00:32:03 to 97/08/31 00:27:26) Session Succ Last Okay Recent Login Link Dial Line Wrong Startups Calls Session Fails Fail Fail Fail Locked Time eci486 12 12 8/30 22:06 0 0 0 0 0 0 gate 8 8 8/30 22:07 0 0 0 0 0 0 UUCP failures and errors (97/08/30 00:32:03 to 97/08/31 00:27:26) ERROR (uux) pid: 4459 (8/30-13:55:48) CAN'T CHMOD /usr/spool/uucp/gate (1) [SCCSID: , FILE expfile.c, LINE 192] -rw------- 1 uucp user 104 Aug 30 1992 X.gate413N164e UUCP neighbors with no recent activity System Name Last OK Blain 97/01/14 Quarta 97/07/12 TOR 97/05/24 ams 97/07/26 cd10 96/12/21 clnat 97/03/28 dvlpment 97/07/04

Back to top
UUCP cleanup report
This report section provides information on System Expert's activity of running the standard UUCP clean up script. This information is valuable to the system administrator responsible for maintaining the UUCP subsystem.
Example:

(reported in blocks) 4 /usr/spool/uucp/.Admin 2 /usr/spool/uucp/.Corrupt 2 /usr/spool/uucp/.Log/uucico 2 /usr/spool/uucp/.Log/uucp 2 /usr/spool/uucp/.Log/uux 2 /usr/spool/uucp/.Log/uuxqt 10 /usr/spool/uucp/.Log 108 /usr/spool/uucp/.Old 8 /usr/spool/uucp/.Sequence 6 /usr/spool/uucp/.Status 2 /usr/spool/uucp/.Workspace 2 /usr/spool/uucp/.Xqtdir 2 /usr/spool/uucp/eci486 2 /usr/spool/uucp/gate 148 /usr/spool/uucp

Back to top
su command usage - URGENT
This report section provides information on the failed 'su' attempts by users to change to important users, such as the 'root, adm, uucp, or news' account. This information is valuable to the security analyst and system administrator responsible for maintaining system integrity and optimal operation of the business system.
Example:

Failed su attempts to important userids Date Time Port From To 08/30 09:32 ttys1 lewis root

Back to top
su command usage
This report section provides information on the failed and successful 'su' attempts by other users to important users, such as the ' root, adm, uucp, news' accounts. This information is valuable to the security analyst and system administrator responsible for maintaining system integrity and optimal operation of the business system.
Example:

lewis -> root 4 09:32-ttys1 Failed 09:34-ttys1 10:46-ttys1 11:38-ttys1 ingres -> root 1 08:45-ttys0

Back to top
Back to top
Line usage and logins
This report section provides information on local and remote usage of serial line devices and by login name. This information is valuable to the security analyst and system administrator responsible for maintaining system security and optimal system performance.
Example:

UUCP incoming logins by login name Off Login Prime Prime Total Name Sessions Time Time Time uibm 3 44 3 47 Failed login attempts by user UNKNOWN_ 6 cd 1 root 1 takada 3 uuaxion 1 Failed login attempts by device
tty1 97/08/23 01:39:02-UNKNOWN_ 01:40:45-UNKNOWN_ 02:39:02-UNKNOWN_ 02:40:45-UNKNOWN_ 04:39:04-UNKNOWN_ 04:41:15-UNKNOWN_ [xoback.yyyy.yy.] 97/08/23 16:31:52-root [xopenj.yyyy.yy.] 97/08/16 08:29:38-akada 08:29:52-akada 08:30:03-akada console 97/08/23 15:50:42-cd m_tb_1 97/08/16 11:08:02-uuaion Non-UUCP logins by login name Off Prime Prime Total Start Time Time Time User Device Time Used Used Used ingres pty/ttys1 97/08/30-08:22:49 32:59 7:11 40:10 patrick pty/ttys1 97/08/30-09:03:02 4:11 0 4:11 ingres pty/ttys1 97/08/30-09:07:18 2:04:07 0 2:04:07 ellent pty/ttys3 97/08/30-09:33:22 1:58:07 0 1:58:07 ingres pty/ttys4 97/08/30-09:47:06 40:39 0 40:39 ingres pty/ttys4 97/08/30-10:27:48 1:07:19 0 1:07:19 ingres pty/ttys1 97/08/30-11:11:33 2:45:19 0 2:45:19 lewis pty/ttys3 97/08/30-11:31:38 5:28:22 2:37 5:30:59 root pty/ttys4 97/08/30-11:35:13 5:24:47 2:37 5:27:24 patrick pty/ttys5 97/08/30-12:13:37 4:46:23 2:37 4:49:00 root pty/ttys1 97/08/30-13:57:05 40:54 0 40:54 root pty/ttys1 97/08/30-14:38:03 2:21:57 2:37 2:24:34

Back to top
Oracle SID HR: Assurance checklist - URGENT
Example:

Assurance item Status ------------------------------------------------------------------ --------- Redo logs mirrored (multiple members) failed** All rollback segments online failed** Optimal shared pool failed**

Back to top
Oracle SID HR: Assurance checklist
Example:

Assurance item Status ------------------------------------------------------------------ --------- Shared cursor ratio > 95% verified Server binary user ownership verified Server binary permissions verified DBA group membership unchanged glogin.sql user/group ownership verified glogin.sql permissions verified Control files on separate devices verified Control file user ownership verified Control file group ownership verified Control file permissions verified Control files meet minimum required number verified Redo logs mirrored (multiple members) failed** Redo group members of same size verified Redo logs on separate devices verified Redo log user ownership verified Redo log group ownership verified Redo log permissions verified Data file user ownership verified Data file group ownership verified Data file permissions verified Object count statistics recorded Tablespace statistics recorded Object free list integrity verified SYS.DUAL table integrity verified _NEXT_OBJECT is the maximum OBJ# in SYS.OBJ$ verified _NEXT_USER is the maximum USER# in SYS.USER$ verified _NEXT_CONSTRAINT is the maximum CON# in SYS.CON$ verified Presence of user SYS in SYS.USER$ verified PUBLIC role present in SYS.USER$ verified DEFAULT profile present in SYS.PROFNAME$ verified No positive values of PRIVILEGE# present verified SYSTEM undo segment online verified SYSTEM undo segment has FILE# 1 and BLOCK# 2 verified All tablespaces online verified All rollback segments online failed** UNDO sequence number < 32000 verified Rollback segment state check verified Objects and their privileges unchanged Objects created/dropped by SYS unchanged Changes in System Privileges unchanged Changes in user objects unchanged Oracle user attributes recorded No large anonymous PL/SQL blocks (>500) verified Optimal shared pool failed** Tablespace freespace statistics recorded

Back to top
Oracle SID HR: Control files - URGENT
Example:

Two or more control files are on the same device (flag D). It is strongly recommended that control files be placed on separate devices. Permissions errors: 3 Found Expected Control file ----- -------- -------------------------------------------------------------- 0660 0640 /oracle/hr/ctrl1HR.ctl 0660 0640 /oracle/hr/ctrlHR.ctl 0660 0640 /oravol/hr/ctrlHR.ctl

Back to top
Oracle SID HR: Control files
Example:

Status Size Control file ------ -------- ------------------------------------------------------------- DP 823.5 Kb /oracle/hr/ctrl1HR.ctl DP 823.5 Kb /oracle/hr/ctrl3HR.ctl P 823.5 Kb /oravol/hr/ctrl2HR.ctl

Back to top
Oracle SID HR: Data file statistics - URGENT
Example:

Permissions errors: 5 Found Expected Data file ----- -------- -------------------------------------------------------------- 0660 0640 /oracle/hr/qwapp.dbf 0660 0640 /oracle/hr/qwapp1.dbf 0660 0640 /oracle/hr/qwapp3.dbf 0660 0640 /oravol/hr/hrapp.dbf

Back to top
Oracle SID HR: Data file statistics
Example:

Status Size Checkpoint F# Data file ------- -------- ---------- -- --------------------------------------------- ONLINE 5.0 Mb 9262528 30 /oracle/hr/XRSA.dbf ONLINE 5.0 Mb 9262528 5 /oracle/hr/qwapp.dbf ONLINE 5.0 Mb 9262528 6 /oracle/hr/qwapp1.dbf ONLINE 5.0 Mb 9262528 7 /oracle/hr/qwapp3.dbf ONLINE 12.0 Mb 9262528 28 /oracle/hr/hrapp1.dbf ONLINE 250.0 Mb 9262528 20 /oracle/hr/hrapp2.dbf ------------------------------------------------------------------------------- Total 732 Mb

Back to top
Oracle SID HR: Performance tuning statistics
Example:

Statistics capture Started: 24-SEP-97 03:21:54 Ended: 25-SEP-97 00:11:05 Statistic Value ------------------------------------ --------------- Consistent gets 8455541 DB block gets 489896 Total gets 8945437 Physical reads 625509 Buffer cache hits 0.9 Physical writes 34900 Redo log space requests 1 Redo entries 58776 Redo size 7081173 Redo log buffer requests 0 Buffer busy waits 0 Free buffer waits 71 Table fetch continued row 62 Table fetch by rowid 415748 Table scan rows gotten 13562123 Free buffer inspected 23092 Free buffer requested 637952 Sorts (disk) 10 Sorts (memory) 89694 Sorts (rows) 607922 User rollbacks 23 User commits 10152 Recursive calls 0 User calls 885394 Parse count 277397 DBWR cross instance writes 0 Summed dirty queue length 1783 DBWR free buffers found 207097 DBWR make free requests 1736 DBWR buffers scanned 210980 DBWR lru scans 1740 Write requests 12080 Continued row access 0 Indexed row access 0.0 DBWR buffers per scan 121.2 DBWR reusable buffers 119. Free buffers inspected 0.0 DBWR ping traffic 0 Sort overflow 0 Rows per sort 6.7 Rollbacks 0 Recursive/user calls 0 Calls per parse 3.1 Redo record size (avg bytes) 1 Redo generation rate (Kb/min) 0 Transaction rate (cmt/min) 0 ------------------------------------------------------------------------------- Pins Reloads Lcache Library cache pins 1111745 967 1 ------------------------------------------------------------------------------- Requested Missed Cached Row cache gets 349830 16542 0.95 ------------------------------------------------------------------------------- trans_waits trans_gets undoWrites rbs_wait Rbs wait 0 22047 2357778 0 ------------------------------------------------------------------------------- Latch misses Misses Gets Ratio Total 930 28033112 0 Immediate 1018 14293858 0 ------------------------------------------------------------------------------- Library - cache statistics Gets GetHits Pins PinHits Reloads Invals ---------- ---------- ---------- ---------- ------- ------ BODY 0 0 0 1 0 0 CLUSTER 27 12 15 0.3 0 0 INDEX 25 0 25 0.0 0 0 OBJECT 0 0 0 1 0 0 PIPE 0 0 0 1 0 0 SQL AREA 277412 274900 1103053 1 870 18 TABLE/PROCEDURE 200197 199749 8652 0.9 97 0 TRIGGER 0 0 0 1 0 0 ------------------------------------------------------------------------------- Dictionary cache (rowcache) statistics GetReqs GetMisses ScanReqs ScanMissd ModReqs Count CurUsage ---------- ---------- ---------- ---------- --------- --------- --------- dc_tablespaces 164 15 0 0 0 7 3 dc_free_extents 1835 60 39 0 77 19 18 dc_segments 12506 1771 0 0 65 10 9 dc_rollback_segments 3566 7 0 0 9 17 8 dc_used_extents 77 41 0 0 77 3 2 dc_tablespace_quotas 13 2 0 0 9 3 2 dc_users 1857 18 0 0 0 15 14 dc_user_grants 1349 16 0 0 0 41 13 dc_objects 200547 339 0 0 59 278 275 dc_tables 16866 728 0 0 38 33 30 dc_columns 68335 9460 5168 834 213 1362 275 dc_table_grants 1257 341 0 0 0 129 32 dc_indexes 9107 1152 4316 634 23 175 21 dc_constraint_defs 16525 2246 642 163 24 19 1 dc_constraint_defs 0 0 477 128 0 1 0 dc_synonyms 57 37 0 0 0 13 12 dc_usernames 733 10 0 0 0 20 10 dc_object_ids 14791 258 0 0 0 230 228 dc_constraints 36 19 0 0 36 20 7 dc_sequences 150 1 0 0 8 2 1 dc_tablespaces 41 3 0 0 41 5 3 dc_histogram_defs 16 16 0 0 0 2 1 dc_truncates 2 2 0 0 0 23 0 ------------------------------------------------------------------------------- Statistic totals Total transactions: 10152 Change per Change per Total logons: 148 Total transaction logon ----------- ----------- ----------- CR blocks created 10707 1.05 72.34 Current blocks converted for CR 13 0 0.09 DBWR buffers scanned 210980 20.78 1426 DBWR checkpoints 41 0 0.28 DBWR free buffers found 207097 20.40 1399 DBWR lru scans 1740 0.17 11.76 DBWR make free requests 1736 0.17 11.73 DBWR summed scan depth 213181 21 1440 DBWR timeouts 18325 1.81 123.82 SQL*Net roundtrips to/from client 885274 87.20 5982 Background checkpoints completed 2 0 0.01 Background checkpoints started 2 0 0.01 Background timeouts 49774 4.90 336.31 Bytes received via SQL*Net from client 113844775 11214 769221 Bytes sent via SQL*Net to client 72374988 7129 489020 Calls to get snapshot scn: kcmgss 294565 29.02 1990 Calls to kcmgas 10275 1.01 69.43 Calls to kcmgcs 375 0.04 2.53 Calls to kcmgrs 343732 33.86 2323 Cleanouts and rollbacks - consistent read gets 8489 0.84 57.36 Cleanouts only - consistent read gets 18580 1.83 125.54 Cluster key scan block gets 27364 2.70 184.89 Cluster key scans 10833 1.07 73.20 Consistent changes 11059 1.09 74.72 Consistent gets 8455541 832.89 57132 Cursor authentications 3131 0.31 21.16 Data blocks consistent reads - undo records applied 11055 1.09 74.70 DB block changes 90038 8.87 608.36 DB block gets 489896 48.26 3310 Deferred (CURRENT) block cleanout applications 193 0.02 1.30 Dirty buffers inspected 535 0.05 3.61 Enqueue conversions 19 0 0.13 Enqueue releases 24816 2.44 167.68 Enqueue requests 24856 2.45 167.95 Enqueue waits 40 0 0.27 Execute count 284923 28.07 1925 Free buffer inspected 23092 2.27 156.03 Free buffer requested 637952 62.84 4310 Global lock gets (non async) 1 0 0.01 Immediate (CR) block cleanout applications 27069 2.67 182.90 Immediate (CURRENT) block cleanout applications 429 0.04 2.90 Logons cumulative 148 0.01 1 Logons current 7 0 0.05 Messages received 18331 1.81 123.86 Messages sent 18331 1.81 123.86 No work - consistent read gets 5352709 527.26 36167 Opened cursors cumulative 114292 11.26 772.24 Opened cursors current 7 0 0.05 Parse count 277397 27.32 1874 Physical reads 625509 61.61 4226 Physical writes 34900 3.44 235.81 Recursive calls 367241 36.17 2481 Redo blocks written 23707 2.34 160.18 Redo buffer allocation retries 6 0 0.04 Redo entries 58776 5.79 397.14 Redo log space requests 1 0 0.01 Redo size 7081173 697.52 47846 Redo small copies 58775 5.79 397.13 Redo synch writes 10218 1.01 69.04 Redo wastage 4593507 452.47 31037 Redo writes 16995 1.67 114.83 Rollback changes - undo records applied 65 0.01 0.44 Rollbacks only - consistent read gets 2232 0.22 15.08 Session logical reads 8932146 879.84 60352 Session pga memory 14933044 1471 100899 Session pga memory max 15279956 1505 103243 Session uga memory 614924 60.57 4155 Session uga memory max 4812332 474.03 32516 Sorts (disk) 10 0 0.07 Sorts (memory) 89694 8.84 606.04 Sorts (rows) 607922 59.88 4108 Summed dirty queue length 1783 0.18 12.05 Table fetch by rowid 415748 40.95 2809 Table fetch continued row 62 0.01 0.42 Table scan blocks gotten 1344527 132.44 9085 Table scan rows gotten 13562123 1336 91636 Table scans (long tables) 34138 3.36 230.66 Table scans (short tables) 107282 10.57 724.88 Transaction rollbacks 5 0 0.03 User calls 885394 87.21 5982 User commits 10152 1 68.59 User rollbacks 23 0 0.16 Write requests 12080 1.19 81.62 ------------------------------------------------------------------------------- System wide events Count WaitTime AvgWait ----------- ----------- ----------- Instance state change 1 0 0 Log file sequential read 5 0 0 Log file single write 6 0 0 Log file space/switch 6 0 0 RDBMS ipc reply 12 0 0 SQL*Net more data from client 17 0 0 Library cache pin 28 0 0 SQL*Net break/reset to client 52 0 0 Enqueue 58 0 0 Free buffer waits 71 0 0 Write complete waits 111 0 0 DB file single write 114 0 0 Control file parallel write 164 0 0 Smon timer 252 0 0 Control file sequential read 618 0 0 Latch free 1657 0 0 DB file parallel write 12080 0 0 Log file sync 16485 0 0 Log file parallel write 16991 0 0 Pmon timer 24989 0 0 RDBMS ipc message 67535 0 0 DB file scattered read 67650 0 0 DB file sequential read 149772 0 0 SQL*Net more data to client 765069 0 0 SQL*Net message from client 885681 0 0 SQL*Net message to client 885683 0 0 ------------------------------------------------------------------------------- Average write queue length QueueChge WritChge AvgWRqueLth 1783 12080 0.15 ------------------------------------------------------------------------------- File I-O spread analysis Tablespace File PhysReads PhysBlkRd PhysRdTime PhysWrites PhysBlksWR PhysWRTime ------------------------------------------------------------------------------- WNAPP /oravol/hr/wnapp.dbf 35 35 0 14 14 0 QWAPP /oracle/hr/qwapp.dbf 0 0 0 0 0 0 QWAPP1 /oracle/hr/qwapp1.dbf 0 0 0 0 0 0 QWAPP3 /oracle/hr/qwapp3.dbf 0 0 0 0 0 0 HRAPP /oracle/hr/hrapp1.dbf 10 10 0 2 2 0 HRAPP /oracle/hr/hrapp2.dbf 29 29 0 13 13 0 HRAPP /oravol/hr/hrapp.dbf 81 81 0 0 0 0 HRWAPP /oravol/hr/hrwapp.dbf 4290 6227 0 0 0 0 HRWAPP /oravol/hr/hrwapp1.dbf 2145 3879 0 0 0 0 HRWAPP /u00/orad/hr/hrwapp3.dbf 0 0 0 0 0 0 HRWAPP /u00/orad/hr/hrwapp4.dbf 0 0 0 0 0 0 HRWAPP /u02/orad/hr/hrwapp2.dbf 2457 3838 0 91 91 0 ------------------------------------------------------------------------------- Tablespace I-O summary PhysReads PhysBlkRd PhysRdTime PhysWrites PhysBlksWR PhysWRTime ---------- ---------- ---------- ---------- ---------- ---------- WNAPP 35 35 0 14 14 0 QWAPP 0 0 0 0 0 0 QWAPP1 0 0 0 0 0 0 QWAPP3 0 0 0 0 0 0 HRAPP 120 120 0 15 15 0 HRWAPP 8892 13944 0 91 91 0 HTAPP 304 618 0 31 31 0 ------------------------------------------------------------------------------- System latches Gets Misses Sleeps HitRatio SleepMisRt ------------ --------- --------- ---------- ---------- Cache buffer handles 20980 0 0 1 0 Cache buffers chains 16584252 68 138 1 2.03 Cache buffers lru chain 708198 72 145 1 2.01 Dml lock allocation 27163 0 0 1 0 Enqueues 63403 2 5 1 2.50 Latch wait list 1541 0 0 1 0 Library cache 5336210 696 1194 1 1.72 Library cache load lock 1134 0 0 1 0 Messages 173041 4 11 1 2.75 Multiblock read objects 140604 3 6 1 2 Process allocation 147 0 0 1 0 Redo allocation 162522 10 8 1 0.80 Row cache objects 817028 30 76 1 2.53 Sequence cache 711 0 0 1 0 Session allocation 25473 0 0 1 0 Session idle bit 1782673 11 25 1 2.27 Session switching 1199 0 0 1 0 Shared pool 1210100 22 25 1 1.14 System commit number 893324 10 20 1 2 Transaction allocation 21755 1 2 1 2 Undo global data 61114 1 2 1 2 User lock 540 0 0 1 0 ------------------------------------------------------------------------------- Statistics on no_wait gets of latches Latch name Gets Misses Sleeps ---------------------------------------- ----------- ----------- ----------- Cache buffers chains 5767624 200 0 Cache buffers lru chain 8524601 816 0 Library cache 727 2 0 Process allocation 147 0 0 Row cache objects 759 0 0 ------------------------------------------------------------------------------- Statistics on rollback I-O Undo Transaction Transaction Undo bytes Segment segment gets waits written bytes Xacts Shrinks Wraps ------- ----------- ----------- ---------- -------- ------ ------- ----- 0 271 0 1.4 Kb 536.0 Kb 0 0 0 2 5417 0 532.1 Kb 2.3 Mb 0 0 1 3 5515 0 726.9 Kb 2.3 Mb 0 0 1 4 5409 0 502.3 Kb 2.3 Mb 0 0 1 5 5435 0 539.8 Kb 2.3 Mb 0 0 1

Back to top
Oracle SID HR: Product versions
Example:

Product Version Status ---------------------------------------- --------- -------------------------- CORE 3.4.3.0.0 Production NLSRTL 3.1.4.6.0 Production Oracle7 Server 7.2.3.0.0 Production Release PL/SQL 2.2.3.0.0 Production TNS for IBM/AIX RISC System/6000: 2.2.3.0.0 Production

Back to top
Oracle SID HR: Redo log file statistics - URGENT
Example:

The online redo log group (1) has only one member. The online redo log group (2) has only one member. The online redo log group (3) has only one member. It is strongly recommended that redo logs be mirrored on separate devices. Permissions errors: 3 Found Expected Redo Log File ----- -------- -------------------------------------------------------------- 0660 0640 /oravol/hr/log01.dbf 0660 0640 /oracle/hr/log02.dbf 0660 0640 /oravol/hr/log03.dbf

Back to top
Oracle SID HR: Redo log file statistics
Example:

Flg Grp Thrd SeqNo Size Arc Status Member --- --- ---- -------- -------- --- -------- ----------------------------- P 1 1 1654 10.0 Mb NO INACTIVE /oravol/hr/log01.dbf ,IN USE P 2 1 1655 10.0 Mb NO INACTIVE /oracle/hr/log02.dbf ,IN USE P 3 1 1656 10.0 Mb NO CURRENT, /oravol/hr/log03.dbf IN USE

Back to top
Oracle SID HR: Recommended shared pool - URGENT
Example:

It is strongly recommended that the shared pool be increased from 8.6 Mb to 12.5 Mb if the following parameters remain the same: No. of users = 3, Max. memory for all sessions = 398.7 Kb, Tolerance=0%

Back to top
Oracle SID HR: Rollback segments statistics
Example:

Rbs name Sizes: Rbs size Opt size HiWatMk Avg actv ----------------------------- -------- -------- -------- -------- R01 Counts: 4 0 1 0 Sizes: 2.3 Kb nav 2.3 Kb 0 R02 Counts: 4 0 1 0 Sizes: 2.3 Kb nav 2.3 Kb 330 R03 Counts: 4 0 1 0 Sizes: 2.3 Kb nav 2.3 Kb 0 R04 Counts: 4 0 1 0 Sizes: 2.3 Kb nav 2.3 Kb 0 SYSTEM Counts: 9 0 0 0 Sizes: 536.0 b nav 536.0 b 0

Back to top
Oracle SID HR: Rollback segment status errors - URGENT
Example:

One or more rollback segments are not online. Status Rollback segment name ------- ---------------------------------------------------------------------- OFFLINE R00 OFFLINE RBSBIG

Back to top
Oracle SID HR: Tablespace free space
Example:

Total Free Used Tablespace bytes bytes % File ---------- -------- -------- ------ --------------------------------------- WNAPP 10.0 Mb 2.6 Mb 73.59 /oravol/hr/wnapp.dbf QWAPP 5.0 Mb 4.2 Mb 16.48 /oracle/hr/qwapp.dbf QWAPP1 5.0 Mb 4.2 Mb 16.48 /oracle/hr/qwapp1.dbf QWAPP3 5.0 Mb 5.0 Mb 0.86 /oracle/hr/qwapp3.dbf HRAPP 12.0 Mb 976.0 Kb 92.06 /oravol/hr/hrapp.dbf 50.0 Mb 35.9 Mb 28.14 /oracle/hr/hrapp2.dbf 12.0 Mb 668.0 Kb 94.56 /oracle/hr/hrapp1.dbf -------- -------- ------ 74.0 Mb 37.5 Mb 49.28 HRWAPP 21.0 Mb 56.0 Kb 99.74 /oravol/hr/hrwapp.dbf 21.0 Mb 880.0 Kb 95.91 /oravol/hr/hrwapp1.dbf 20.0 Mb 3.8 Mb 81.07 /u02/orad/hr/hrwapp2.dbf 32.0 Kb 28.0 Kb 12.50 /u00/orad/hr/hrwapp3.dbf 40.0 Mb 40.0 Mb 0.01 /u00/orad/hr/hrwapp4.dbf -------- -------- ------ 102.0 Mb 44.7 Mb 56.17 HTAPP 8.0 Mb 344.0 Kb 95.80 /oravol/hr/htapp.dbf 8.0 Mb 8.0 Mb 0.05 /oravol/hr/htapp1.dbf -------- -------- ------ 16.0 Mb 8.3 Mb 47.92

Back to top
Oracle SID HR: Tablespace statistics
Example:

Initial Next Min Max % Tablespace name extent extent extent extent incrs Status --------------------------- -------- -------- ------ ------ ----- ------- WNAPP 20.0 Kb 20.0 Kb 1 249 50 ONLINE QWAPP 64.0 Kb 128.0 Kb 1 110 0 ONLINE QWAPP1 64.0 Kb 128.0 Kb 1 110 0 ONLINE QWAPP3 64.0 Kb 128.0 Kb 1 110 0 ONLINE HRAPP 64.0 Kb 128.0 Kb 1 110 0 ONLINE HRWAPP 64.0 Kb 128.0 Kb 1 110 0 ONLINE