Commands Reference, Volume 5

trace Daemon

Purpose

Records selected system events.

Syntax

trace [ -a [ -g ] ] [ -f | -l ] [ -b | -B] [ -c] [ -C [ CPUList | all ]] [ -d ] [ -h ] [ -j Event [ ,Event ] ] [ -k Event [ ,Event ] ] [ -J Event-group [ ,Event-group ]] [ -K Event-group [ ,Event-group ]] [ -m Message ] [ -n ] [ -o Name ] [ -o- ] [ -p ] [ -s ] [ -L Size ] [ -T Size ]

Description

The trace daemon configures a trace session and starts the collection of system events. The data collected by the trace function is recorded in the trace log. A report from the trace log can be generated with the trcrpt command.

When invoked with the -a flag, the trace daemon is run asynchronously (i.e. as a background task). Otherwise, it is run interactively and prompts you for subcommands.

You can use the System Management Interface Tool (SMIT) to run the trace daemon. To use SMIT, enter:

smit trace

There are three modes of trace data collection:

Alternate (the default)	All trace events are captured in the trace log file.
Circular ( -l)	The trace events wrap within the in-memory buffers and are not captured in the trace log file until the trace data collection is stopped.
Single ( -f)	The collection of trace events stops when the in-memory trace buffer fills up and the contents of the buffer are captured in the trace log file.
Buffer Allocation	Trace buffers are allocated from either the kernel heap, or are put into separate segments. By default, buffers are allocated from the kernel heap unless the buffer size requested is too large for buffers to fit in the kernel heap, in which case they are allocated in separate segments. Allocating buffers from separate segments hinders trace performance somewhat. However, buffers in separate segments will not take up paging space, just pinned memory. The type of buffer allocation can be specified with the optional -b or -B flags.

Flags

-a	Runs the trace daemon asynchronously (i.e. as a background task). Once trace has been started this way, you can use the trcon, trcoff, and trcstop commands to respectively start tracing, stop tracing, or exit the trace session. These commands are implemented as links to trace.
-b	Allocate buffers from the kernel heap. If the requested buffer space can not be obtained from the kernel heap, the command fails. Note The -b flag is only valid with the 32-bit kernel.
-B	Allocate buffers in separate segments. Note The -B flag is only valid with the 32-bit kernel.
-c	Saves the trace log file, adding .old to its name.
-C [ CPUList \| all ]	Traces using one set of buffers per CPU in the CPUList. The CPUs can be separated by commas, or enclosed in double quotation marks and separated by commas or blanks. To trace all CPUs, specify all. Since this flag uses one set of buffers per CPU, and produces one file per CPU, it can consume large amounts of memory and file space, and should be used with care. The files produced are named trcfile, trcfile-0, trcfile-1, etc., where 0, 1, etc. are the CPU numbers. If -T or -L are specified, the sizes apply to each set of buffers and each file. On a uniprocessor system, you may specify `-C all`, but `-C` with a list of CPU numbers is ignored. If -C is used to specify more than one CPU, such as `-Call` or `-C "0 1"`, the associated buffers are not put into the system dump.
-d	Disables the automatic start of trace data collection. Delays starting of trace data collection. Normally, the collection of trace data starts automatically when you issue the trace daemon. Use the trcon command to start the collection of trace data.
-f	Runs trace in a single mode. Causes the collection of trace data to stop as soon as the in-memory buffer is filled up. The trace data is then written to the trace log. Use the trcon command to restart trace data collection and capture another full buffer of data. If you issue the trcoff subcommand before the buffer is full, trace data collection is stopped and the current contents of the buffer are written to the trace log.
-g	Starts a trace session on a generic trace channel (channels 1 through 7). This flag works only when trace is run asynchronously (-a). The return code of the command is the channel number; the channel number must subsequently be used in the generic trace subroutine calls. To stop the generic trace session, use the command trcstop -<channel_number>.
-h	Omits the header record from the trace log. Normally, the tracedaemon writes a header record with the date and time (from the date command) at the beginning of the trace log; the system name, version and release, the node identification, and the machine identification (from the uname -a command); and a user-defined message. At the beginning of the trace log, the information from the header record is included in the output of the trcrpt command.
-j Event[,Event]
-k Event[,Event]	Specifies the user-defined events for which you want to collect (-j) or exclude (-k) trace data. The Event list items can be separated by commas, or enclosed in double quotation marks and separated by commas or blanks. Note: The following events are used to determine the pid, the cpuid and the exec path name in the trcrpt report: 106 DISPATCH 10C DISPATCH IDLE PROCESS 134 EXEC SYSTEM CALL 139 FORK SYSTEM CALL 465 KTHREAD CREATE If any of these events is missing, the information reported by the trcrpt command will be incomplete. Consequently: when using the -j flag, you should include all these events in the Event list; conversely, when using the -k flag, you should not include these events in the Event list. If starting the trace with SMIT, or the -J flag, these events are in the tidhk group.
-J Event-group [, Event-group ]
-K Event-group [ ,Event-group ]	Specifies the event groups to be included (-J) or excluded (-K). Event groups are described in "Debug and Performance Tracing". The -J and -K flags work like -j and -k, except with event groups instead of individual hook ids. All four flags, -j, -J, -k, and -K may be specified.
-l	Runs trace in a circular mode. The trace daemon writes the trace data to the trace log when the collection of trace data is stopped. Only the last buffer of trace data is captured. When you stop trace data collection using the trcoff command, restart it using the trconcommand.
-L Size	Overrides the default trace log file size of 1MB with the value stated. Specifying a file size of zero sets the trace log file size to the default size. HERE Note: In the circular and the alternate modes, the trace log file size must be at least twice the size of the trace buffer. In the single mode, the trace log file must be at least the size of the buffer. See the -T flag for information on controlling the trace buffer size.
-m Message	Specifies text to be included in the message field of the trace log header record.
-n	Adds information to the trace log header: lock information, hardware information, and, for each loader entry, the symbol name, address, and type.
-o Name	Overrides the /var/adm/ras/trcfile default trace log file and writes trace data to a user-defined file.
-o -	Overrides the default trace log name and writes trace data to standard output. The -c flag is ignored when using this flag. An error is produced if -o - and -C are specified.
-p	Includes the cpuid of the current processor with each hook. This flag is only valid for 64-bit kernel traces. Note: The trcrpt command can report the cpuid whether or not this option is specified.
-s	Stops tracing when the trace log fills. The trace daemon normally wraps the trace log when it fills up and continues to collect trace data. During asynchronous operation, this flag causes the trace daemon to stop trace data collection. (During interactive operation, the quit subcommand must be used to stop trace.)
-T Size	Overrides the default trace buffer size of 128KB with the value stated. You must be root to request more than 1MB of buffer space. The maximum possible size is 268435184 bytes, unless the -f flag is used, in which case it is 536870368 bytes. The smallest possble size is 8192 bytes, unless the -f flag is used, in which case it is 16392 bytes. Sizes between 8192 and 16392 will be accepted when using the -f flag; however, the actual size used will be 16392 bytes. Note: In the circular and the alternate modes, the trace buffer size must be one-half or less the size of the trace log file. In the single mode, the trace log file must be at least the size of the buffer. See the -L flag for information on controlling the trace log file size. Also note that trace buffers use pinned memory, which means they are not pageable. Therefore, the larger the trace buffers, the less physical memory is available to applications. Unless the -b or -B flags are specified, the system attempts to allocate the buffer space from the kernel heap. If this request can not be satisfied, the system then attempts to allocate the buffers as separate segments. The -f flag actually uses two buffers, which behave as a single buffer (except that a buffer wraparound trace hook will be recorded when the first buffer is filled).

Subcommands

When run interactively, trace recognizes the following subcommands:

trcon	Starts the collection of trace data.
trcoff	Stops the collection of trace data.
q or quit	Stops the collection of trace data and exits trace.
! Command	Runs the shell command specified by the Command parameter.
?	Displays the summary of trace subcommands.

Signals

The INTERRUPT signal acts as a toggle to start and stop the collection of trace data. Interruptions are set to SIG_IGN for the traced process.

Examples

To use trace interactively, enter trace, (the > prompt is displayed), then specify the subcommands you want. For example, to trace system events during the run of the anycmd command, enter:
```
trace
> !anycmd
> q
```
To avoid delays when the command finishes, start trace asynchronously ( -a), using only one command line. Enter:
```
trace -a; anycmd; trcstop
```
To trace the system itself for a period of 10 seconds, enter:
```
trace -a; sleep 10; trcstop
```
To output trace data to a specific trace log file (instead of the /var/adm/ras/trcfile default trace log file), :
```
trace -a -o /tmp/my_trace_log; anycmd; trcstop
```
To capture the execution of a cp command, excluding specific events from the collection process:
```
trace -a -k "20e,20f"; cp /bin/track /tmp/junk; trcstop
```
In the example above, the -k option suppresses the collection of events from the lockl and unlockl functions (20e and 20f events).
To trace hook 234 and the hooks that will allow you to see the process names, use:
```
trace -a -j 234 -J tidhk
```
This traces the hooks in the event-group "tidhk" plus hook 234.
To have trace use one set of buffers per CPU, specify:
```
trace -aC all
```
The files produced are /var/adm/ras/trcfile, /var/adm/ras/trcfile-0, /var/adm/ras/trcfile-1, etc. up to /var/adm/ras/trcfile-(n-1), where n is the number of CPUs in the system.
Note

trace -aCall -o mylog produces the files mylog, mylog-0, mylog-1, ...

Files

/usr/include/sys/trcmacros.h	Defines trchook and utrchook macros.
/var/adm/ras/trcfile	Contains the default trace log file.

Related Information

The trcevgrp command, the trcnm command, the trcrpt command, the trcstop command.

The trchook subroutine, trcgen subroutine, trcstart subroutine, trcon subroutine, trcoff subroutine, trcstop subroutine.

The trcgenk kernel service.

Trace Facility Overview in AIX 5L Version 5.2 General Programming Concepts: Writing and Debugging Programs

Performance Analysis with the Trace Facility in AIX 5L Version 5.2 Performance Management Guide.

Debug and Performance Tracing in AIX 5L Version 5.2 Kernel Extensions and Device Support Programming Concepts.