Technical Reference: Base Operating System and Extensions, Volume 1

ptrace, ptracex, ptrace64 Subroutine

Purpose

Traces the execution of another process.

Library

Standard C Library (libc.a)

Syntax

#include <sys/reg.h>
#include <sys/ptrace.h>
#include <sys/ldr.h>
int ptrace ( Request, Identifier, Address, Data, Buffer)
int Request;
int Identifier;
int *Address;
int Data;
int *Buffer;

int ptracex ( request, identifier, addr, data, buff)
int request;
int identifier;
long long addr;
int data;
int *buff;

int ptrace64 ( request, identifier, addr, data, buff)
int request;
long long identifier;
long long addr;
int data;
int *buff;

Description

The ptrace subroutine allows a 32-bit process to trace the execution of another process. The ptrace subroutine is used to implement breakpoint debugging.

A debugged process executes normally until it encounters a signal. Then it enters a stopped state and its debugging process is notified with the wait subroutine.

Exception: If the process encounters the SIGTRAP signal, a signal handler for SIGTRAP exists, and fast traps (Fast Trap Instructions) have been enabled for the process, then the signal handler is called and the debugger is not notified. This exception only applies to AIX 4.3.3 and later releases.

While the process is in the stopped state, the debugger examines and modifies the memory image of the process being debugged by using the ptrace subroutine. For multi-threaded processes, the getthrds (getthrds Subroutine) subroutine identifies each kernel thread in the debugged process. Also, the debugging process can cause the debugged process to terminate or continue, with the possibility of ignoring the signal that caused it to stop.

As a security measure, the ptrace subroutine inhibits the set-user-ID facility on subsequent exec subroutines.

(This paragraph only applies to AIX 4.3.1 and previous releases.) When a process is executing under ptrace control, portions of the process's address space are recopied after load, unload, and loadbind calls. For a 32-bit process, the main program text (loaded in segment 1) and shared library modules (loaded in segment 13) is recopied. Any breakpoints or other modifications to these segments must be reinserted after load, unload, or loadbind. Changes to privately loaded modules persist. For a 64-bit process, shared library modules are recopied after load and unload are called. (For AIX 4.3 and AIX 4.3.1, these segments have a virtual address of 0x09000000xxxxxxxx, where x denotes any value.) The segments for the main programs and the segments containing privately loaded modules are not recopied. When a 64-bit process calls loadbind, no segments are recopied and the debugger is not notified.

(This paragraph only applies to AIX 4.3.2 and later releases.) When a process executing under ptrace control calls load or unload, the debugger is notified and the W_SLWTED flag is set in the status returned by wait. (A 32-bit process calling loadbind is stopped as well.) If the process being debugged has added modules in the shared library to its address space, the modules are added to the process's private copy of the shared library segments. If shared library modules are removed from a process's address space, the modules are deleted from the process's private copy of the library text segment by freeing the pages that contain the module. No other changes to the segment are made, and existing breakpoints do not have to be reinserted.

To allow a debugger to generate code more easily (in order to handle fast trap instructions, for example), memory from the end of the main program up to the next segment boundary can be modified. That memory is read-only to the process but can be modified by the debugger.

When a process being traced forks, the child process is initialized with the unmodified main program and shared library segment, effectively removing breakpoints in these segments in the child process. If multiprocess debugging is enabled, new copies of the main program and shared library segments are made. Modifications to privately loaded modules, however, are not affected by a fork. These breakpoints will remain in the child process, and if these breakpoints are executed, a SIGTRAP signal will be generated and delivered to the process.

If a traced process initiates an exec subroutine, the process stops before executing the first instruction of the new image and returns the SIGTRAP signal.

Note: ptrace and ptracex are not supported in 64-bit mode.

Fast Trap Instructions

Note: The "Fast Trap Instructions" section only applies to AIX 4.3.3 and later releases.

Sometimes, allowing the process being debugged to handle certain trap instructions is useful, instead of causing the process to stop and notify the debugger. You can use this capability to patch running programs or programs whose source codes are not available. For a process to use this capability, you must enable fast traps, which requires you to make a ptrace call from a debugger on behalf of the process.

To let a process handle fast traps, a debugger uses the ptrace (PT_SET, pid, 0, PTFLAG_FAST_TRAP, 0) subroutine call. Cancel this capability with the ptrace (PT_CLEAR, pid, 0, PTFLAG_FAST_TRAP, 0) subroutine call. If a process is able to handle fast traps when the debugger detaches, the fast trap capability remains in effect. Consequently, when another debugger attaches to that process, fast trap processing is still enabled. When no debugger is attached to a process, SIGTRAP signals are handled in the same manner, regardless of whether fast traps are enabled.

A fast trap instruction is an unconditional trap immediate instruction in the form twi 14,r13,0xNXXX. This instruction has the binary form 0x0ddfNXXX, where N is a hex digit >=8 and XXX are any three hex digits. By using different values of 0xNXXX, a debugger can generate different fast trap instructions, allowing a signal handler to quickly determine how to handle the signal. (The fast trap instruction is defined by the macro _PTRACE_FASTTRAP. The _PTRACE_FASTTRAP_MASK macro can be used to check whether a trap is a fast trap.)

Usually, a fast trap instruction is treated like any other trap instruction. However, if a process has a signal handler for SIGTRAP, the signal is not blocked, and the fast trap capability is enabled, then the signal handler is called and the debugger is not notified.

A signal handler can logically AND the trap instruction with _PTRACE_FASTTRAP_NUM (0x7FFF) to obtain an integer identifying which trap instruction was executed.

For the 64-bit Process

Use ptracex where the debuggee is a 64-bit process and the operation requested uses the third (Address) parameter to reference the debuggee's address space or is sensitive to register size. Use ptrace64 if the second parameter (identifier) is a thread id from a 64-bit process. Note that ptracex and ptrace64 will also support 32-bit debugees.

If returning or passing an int doesn't work for a 64-bit debuggee (for example, PT_READ_GPR), the buffer parameter takes the address for the result. Thus, with the ptracex subroutine, PT_READ_GPR and PT_WRITE_GPR take a pointer to an 8 byte area representing the register value.

In general, ptracex supports all the calls that ptrace does when they are modified for any that are extended for 64-bit addresses (for example, GPRs, LR, CTR, IAR, and MSR). Anything whose size increases for 64-bit processes must be allowed for in the obvious way (for example, PT_REGSET must be an array of long longs for a 64-bit debuggee).

Parameters

Request

Determines the action to be taken by the ptrace subroutine and has one of the following values:

PT_ATTACH

This request allows a debugging process to attach a current process and place it into trace mode for debugging. This request cannot be used if the target process is already being traced. The Identifier parameter is interpreted as the process ID of the traced process. The Address, Data, and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one the following codes:

ESRCH

Process ID is not valid; the traced process is a kernel process; the process is currently being traced; or, the debugger or traced process already exists.

EPERM

Real or effective user ID of the debugger does not match that of the traced process, or the debugger does not have root authority.

EINVAL

The debugger and the traced process are the same.

 

PT_CLEAR

This request clears an internal flag or capability. The Data parameter specifies which flags to clear. The following flag can be cleared:

PTFLAG_FAST_TRAP

Disables the special handling of a fast trap instruction (Fast Trap Instructions). This allows all fast trap instructions causing an interrupt to generate a SIGTRAP signal.

The Identifier parameter specifies the process ID of the traced process. The Address parameter, Buffer parameter, and the unused bits in the Data parameter are reserved for future use and should be set to 0.

 

PT_CONTINUE

This request allows the process to resume execution. If the Data parameter is 0, all pending signals, including the one that caused the process to stop, are concealed before the process resumes execution. If the Data parameter is a valid signal number, the process resumes execution as if it had received that signal. If the Address parameter equals 1, the execution continues from where it stopped. If the Address parameter is not 1, it is assumed to be the address at which the process should resume execution. Upon successful completion, the value of the Data parameter is returned to the debugging process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The signal to be sent to the traced process is not a valid signal number.

Note: For the PT_CONTINUE request, use ptracex or prtrace64 with a 64-bit debuggee because the resume address needs 64 bits.

 

PTT_CONTINUE

This request asks the scheduler to resume execution of the kernel thread specified by Identifier. This kernel thread must be the one that caused the exception. The Data parameter specifies how to handle signals:

• If the Data parameter is 0, the kernel thread which caused the exception will be resumed as if the signal never occurred.
• If the Data parameter is a valid signal number, the kernel thread which caused the exception will be resumed as if it had received that signal.

The Address parameter specifies where to resume execution:

• If the Address parameter is 1, execution resumes from the address where it stopped.
• If the Address parameter contains an address value other than 1, execution resumes from that address.

The Buffer parameter should point to a PTTHREADS structure, which contains a list of kernel thread identifiers to be started. This list should be NULL terminated if it is smaller than the maximum allowed.

On successful completion, the value of the Data parameter is returned to the debugging process. On unsuccessful completion, the value -1 is returned, and the errno global variable is set as follows:

EINVAL

The Identifier parameter names the wrong kernel thread.

EIO

The signal to be sent to the traced kernel thread is not a valid signal number.

ESRCH

The Buffer parameter names an invalid kernel thread. Each kernel thread in the list must be stopped and belong to the same process as the kernel thread named by the Identifier parameter.

Note: For the PTT_CONTINUE request, use ptracex or ptrace64 with a 64-bit debuggee because the resume address needs 64 bits.

 

PT_DETACH

This request allows a debugged process, specified by the Identifier parameter, to exit trace mode. The process then continues running, as if it had received the signal whose number is contained in the Data parameter. The process is no longer traced and does not process any further ptrace calls. The Address and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

Signal to be sent to the traced process is not a valid signal number.

 

PT_KILL

This request allows the process to terminate the same way it would with an exit subroutine.

PT_LDINFO

This request retrieves a description of the object modules that were loaded by the debugged process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored. The Address parameter specifies the location where the loader information is copied. The Data parameter specifies the size of this area. The loader information is retrieved as a linked list of ld_info structures. The ld_info structures are defined in the /usr/include/sys/ldr.h file. The linked list is implemented so that the ldinfo_nxt field of each element gives the offset of the next element from this element. The ldinfo_nxt field of the last element has the value 0.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

ENOMEM

Either the area is not large enough to accommodate the loader information, or there is not enough memory to allocate an equivalent buffer in the kernel.

Note: For the PT_LDINFO request, use ptracex or ptrace64 with a 64-bit debuggee because the source address needs 64 bits.

 

PT_MULTI

This request turns multiprocess debugging mode on and off, to allow debugging to continue across fork and exec subroutines. A 0 value for the Data parameter turns multiprocess debugging mode off, while all other values turn it on. When multiprocess debugging mode is in effect, any fork subroutine allows both the traced process and its newly created process to trap on the next instruction. If a traced process initiated an exec subroutine, the process stops before executing the first instruction of the new image and returns the SIGTRAP signal. The Identifier parameter is interpreted as the process ID of the traced process. The Address and Buffer parameters are ignored.

Also, when multiprocess debugging mode is enabled, the following values are returned from the wait subroutine:

W_SEWTED

Process stopped during execution of the exec subroutine.

W_SFWTED

Process stopped during execution of the fork subroutine.

 

PT_READ_BLOCK

This request reads a block of data from the debugged process address space. The Address parameter points to the block of data in the process address space, and the Data parameter gives its length in bytes. The value of the Data parameter must not be greater than 1024. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter points to the location in the debugging process address space where the data is copied. Upon successful completion, the ptrace subroutine returns the value of the Data parameter.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one of the following codes:

EIO

The Data parameter is less than 1 or greater than 1024.

EIO

The Address parameter is not a valid pointer into the debugged process address space.

EFAULT

The Buffer parameter does not point to a writable location in the debugging process address space.

Note: For the PT_READ_BLOCK request, use ptracex or ptrace64 with a 64-bit debuggee because the source address needs 64 bits.

 

PT_READ_FPR

This request stores the value of a floating-point register into the location pointed to by the Address parameter. The Data parameter specifies the floating-point register, defined in the sys/reg.h file for the machine type on which the process is executed. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Data parameter is not a valid floating-point register. The Data parameter must be in the range 256-287.

 

PTT_READ_FPRS

This request writes the contents of the 32 floating point registers to the area specified by the Address parameter. This area must be at least 256 bytes long. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

PT_READ_GPR

This request returns the contents of one of the general-purpose or special-purpose registers of the debugged process. The Address parameter specifies the register whose value is returned. The value of the Address parameter is defined in the sys/reg.h file for the machine type on which the process is executed. The Identifier parameter is interpreted as the process ID of the traced process. The Data and Buffer parameters are ignored. The buffer points to long long target area.

Note: If ptracex or ptrace64 with a 64-bit debuggee is used for this request, the register value is instead returned to the 8-byte area pointed to by the buffer pointer.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Address is not a valid general-purpose or special-purpose register. The Address parameter must be in the range 0-31 or 128-136.

 

PTT_READ_GPRS

This request writes the contents of the 32 general purpose registers to the area specified by the Address parameter. This area must be at least 128 bytes long.

Note: If ptracex or ptrace64 are used with a 64-bit debuggee for the PTT_READ_GPRS request, there must be at least a 256 byte target area. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

PT_READ_I or PT_READ_D

These requests return the word-aligned address in the debugged process address space specified by the Address parameter. On all machines currently supported by AIX Version 4, the PT_READ_I and PT_READ_D instruction and data requests can be used with equal results. The Identifier parameter is interpreted as the process ID of the traced process. The Data parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Address is not word-aligned, or the Address is not valid. User blocks, kernel segments, and kernel extension segments are not considered as valid addresses.

Note: For the PT_READ_I or the PT_READ_D request, use ptracex or ptrace64 with a 64-bit debuggee because the source address needs 64 bits.

 

PTT_READ_SPRS

This request writes the contents of the special purpose registers to the area specified by the Address parameter, which points to a ptsprs structure. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

Note: For the PTT_READ_SPRS request, use ptracex or ptrace64 with the 64-bit debuggee because the new ptxsprs structure must be used.

PT_REATT

This request allows a new debugger, with the proper permissions, to trace a process that was already traced by another debugger. The Identifier parameter is interpreted as the process ID of the traced process. The Address, Data, and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one the following codes:

ESRCH

The Identifier is not valid; or the traced process is a kernel process.

EPERM

Real or effective user ID of the debugger does not match that of the traced process, or the debugger does not have root authority.

EINVAL

The debugger and the traced process are the same.

 

PT_REGSET

This request writes the contents of all 32 general purpose registers to the area specified by the Address parameter. This area must be at least 128 bytes for the 32-bit debuggee or 256 bytes for the 64-bit debuggee. The Identifier parameter is interpreted as the process ID of the traced process. The Data and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Address parameter points to a location outside of the allocated address space of the process.

Note: For the PT_REGSET request, use ptracex or ptrace64 with the 64-bit debuggee because 64-bit registers requiring 256 bytes are returned.

 

PT_SET

This request sets an internal flag or capability. The Data parameter indicates which flags are set. The following flag can be set:

PTFLAG_FAST_TRAP

Enables the special handling of a fast trap instruction (Fast Trap Instructions). When a fast trap instruction is executed in a process that has a signal handler for SIGTRAP, the signal handler will be called even if the process is being traced.

The Identifier parameter specifies the process ID of the traced process. The Address parameter, Buffer parameter, and the unused bits in the Data parameter are reserved for future use and should be set to 0.

 

PT_TRACE_ME

This request must be issued by the debugged process to be traced. Upon receipt of a signal, this request sets the process trace flag, placing the process in a stopped state, rather than the action specified by the sigaction subroutine. The Identifier, Address, Data, and Buffer parameters are ignored. Do not issue this request if the parent process does not expect to trace the debugged process.

As a security measure, the ptrace subroutine inhibits the set-user-ID facility on subsequent exec subroutines, as shown in the following example:

if((childpid = fork()) == 0)
{ /* child process */
 ptrace(PT_TRACE_ME,0,0,0,0);
   execlp(          )/* your favorite exec*/
   }
else
{    /* parent                          */
     /* wait for child to stop          */
        rc = wait(status) 

Note: This is the only request that should be performed by the child. The parent should perform all other requests when the child is in a stopped state.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

ESRCH

Process is debugged by a process that is not its parent.

 

PT_WRITE_BLOCK

This request writes a block of data into the debugged process address space. The Address parameter points to the location in the process address space to be written into. The Data parameter gives the length of the block in bytes, and must not be greater than 1024. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter points to the location in the debugging process address space where the data is copied. Upon successful completion, the value of the Data parameter is returned to the debugging process.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one of the following codes:

EIO

The Data parameter is less than 1 or greater than 1024.

EIO

The Address parameter is not a valid pointer into the debugged process address space.

EFAULT

The Buffer parameter does not point to a readable location in the debugging process address space.

Note: For the PT_WRITE_BLOCK request, use ptracex or ptrace64 with the 64-bit debuggee because 64-bit registers requiring 256 bytes are returned.

 

PT_WRITE_FPR

This request sets the floating-point register specified by the Data parameter to the value specified by the Address parameter. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Data parameter is not a valid floating-point register. The Data parameter must be in the range 256-287.

 

PTT_WRITE_FPRS

This request updates the contents of the 32 floating point registers with the values specified in the area designated by the Address parameter. This area must be at least 256 bytes long. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

PT_WRITE_GPR

This request stores the value of the Data parameter in one of the process general-purpose or special-purpose registers. The Address parameter specifies the register to be modified. Upon successful completion, the value of the Data parameter is returned to the debugging process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

Note: If ptracex or ptrace64 are used with a 64-bit debuggee for the PT_WRITE_GPR request, the new register value is NOT passed via the Data parameter, but is instead passed via the 8-byte area pointed to by the buffer parameter.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Address parameter is not a valid general-purpose or special-purpose register. The Address parameter must be in the range 0-31 or 128-136.

 

PTT_WRITE_GPRS

This request updates the contents of the 32 general purpose registers with the values specified in the area designated by the Address parameter. This area must be at least 128 bytes long. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

Note: For the PTT_WRITE_GPRS request, use ptracex or ptrace64 with the 64-bit debuggee because 64-bit registers requiring 256 bytes are returned. The buffer points to long long source area.

PT_WRITE_I or PT_WRITE_D

These requests write the value of the Data parameter into the address space of the debugged process at the word-aligned address specified by the Address parameter. On all machines currently supported by AIX Version 4, instruction and data address spaces are not separated. The PT_WRITE_I and PT_WRITE_D instruction and data requests can be used with equal results. Upon successful completion, the value written into the address space of the debugged process is returned to the debugging process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO

The Address parameter points to a location in a pure procedure space and a copy cannot be made; the Address is not word-aligned; or, the Address is not valid. User blocks, kernel segments, and kernel extension segments are not considered valid addresses.

Note: For the or PT_WRITE_I or PT_WRITE_D request, use ptracex or ptrace64 with a 64-bit debuggee because the target address needs 64 bits.

 

PTT_WRITE_SPRS

This request updates the special purpose registers with the values in the area specified by the Address parameter, which points to a ptsprs structure. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

Identifier

Determined by the value of the Request parameter.

Address

Determined by the value of the Request parameter.

Data

Determined by the value of the Request parameter.

Buffer

Determined by the value of the Request parameter.

Note: For the PTT_READ_SPRS request, use ptracex or ptrace64 with the 64-bit debuggee because the new ptxsprs structure must be used.

 

Error Codes

The ptrace subroutine is unsuccessful when one of the following is true:

For ptrace: If the debuggee is a 64-bit process, the options that refer to GPRs or SPRs fail with errno = EIO, and the options that specify addresses are limited to 32-bits.

For ptracex or ptrace64: If the debuggee is a 32-bit process, the options that refer to GPRs or SPRs fail with errno = EIO, and the options that specify addresses in the debuggee's address space that are larger than 2**32 - 1 fail with errno set to EIO.

Also, the options PT_READ_U and PT_WRITE_U are not supported if the debuggee is a 64-bit program (errno = ENOTSUP).

Implementation Specifics

This subroutine is part of Base Operating System (BOS) Runtime.

Related Information

The exec (exec: execl, execle, execlp, execv, execve, execvp, or exect Subroutine) subroutine, getprocs (getprocs Subroutine) subroutine, getthrds (getthrds Subroutine) subroutine, load (load Subroutine) subroutine, sigaction subroutine, unload subroutine, wait, waitpid, or wait3 subroutine.

The dbx command.