Date: July 16, 1999
Monitoring user activity is an important system administration task. Here are a few useful AIX commands for monitoring user activity.
| Command | Description |
|---|---|
| last | Display information about previous logins, including login date/time, logout time, originating remote host. |
| who | List users who are currently logged on. |
| ps -fu "userid" | List processes being run by "userid". |
| .sh_history | Text file located in the home directory of each Korn shell user. The file contains the last 50 commands issued by the user. |
| /var/adm/sulog | Text file containing all "su" activity. |
| /etc/security/failedlogin | Binary file containing a listing of all failed login attempts. The file can be read by only root, using "who failedlogin" |
AIX has two other facilities for monitoring user activity on a more granular level. The first is "Accounting" which monitors user's connect time and system usage (cpu by command, diskstorage and printer usage). The second is "Auditing", a security related function, which provides a detailed audit trail of each user's activity, including priviledge failures, commands run, files they view/create/delete, and more. Both Accounting and Auditing are part of the base AIX operating system. However, both must be configured to run. The attachment HTML files discuss how to set up accounting and auditing. See your AIX documentation for more information.
AIX System Security Audit [audit.all.cmd]
AIX System Security Audit
-------------------------------------------------------------------------------
Contents
About This Document
Related Documentation
Overview
Auditing Events and Objects
Auditing Mode: BIN and STREAM
Starting and Stopping Audit
Auditing Configuration
Auditing a User
Auditing an Object
Disk Space Consideration
Understanding the Output
Common Problems with Auditing
Data Overload
Fixes for Auditing Subsystem
-------------------------------------------------------------------------------
About This Document
This document is intended to simplify the use of the auditing system provided
in AIX and applies to all versions of AIX. It includes information on what
auditing offers, what its requirements are, and what common problems may be
encountered. The intention is not to answer every question about auditing, but
to provide a starting point for understanding and setting up auditing.
Related Documentation
Documentation for System Auditing can be found in Chapter 5 of the "System
Management Guide" for AIX version 3.2, Chapter 3 of the "System Management
Guide: Operating System and Devices" for AIX version 4, and Chapter 8 in the
IBM Redbook "Elements of Security: AIX 4.1" (GG24-4433-00).
The AIX and RS/6000 product documentation library is also available:
http://www.rs6000.ibm.com/resource/
-------------------------------------------------------------------------------
Overview
The auditing subsystem provides the means to record security-related
information and to alert system administrators of potential and actual
violations of the system security policy. The information collected by auditing
includes: the name of the auditable event, the status (success or failure) of
the event, and any additional event-specific information related to security
auditing.
-------------------------------------------------------------------------------
Auditing Events and Objects
A list of audit events built into AIX, along with a list of predefined audit
objects, can be found in the file /etc/security/audit/events.
In general, auditing events are defined at the system call level. A single
operation at the command line would result in records of several events in the
audit trail. For example, when viewing a file using the cat or more command,
you would see the following records logged into the audit trail:
FILE_Open (file is opened)
FILE_Read (file is read)
FILE_Write (file is written to standard output)
PROC_Create (process creation for more OR cat)
PROC_Execute (command execution)
PROC_Delete (process completion)
Auditing all possible events can produce a huge amount of data. Through audit
controls (that is, modifying the configuration files), you can select events to
be recorded.
Audit events are grouped into classes. The events can be defined by which
events are in a class. While the class names are arbitrary, they, rather than
individual event names, are associated with user IDs when the audit subsystem
is active.
Auditing objects are just individual files that will be monitored. Three
operations can be audited: read, write, and execute. Objects are not associated
with user IDs. Audit records are generated whenever an audited object is
referenced by any user (including root).
To add further audit objects, extend the /etc/security/audit/objects file.
-------------------------------------------------------------------------------
Auditing Mode: BIN and STREAM
There are two modes of operation for auditing: BIN and STREAM. BIN mode writes
the audit trail to alternating temporary files (bins), then to a single trail
file. STREAM mode writes to a circular buffer that is read synchronously
through an audit pseudo-device (/dev/audit).
An audit can be started in one OR both of these modes.
Using the audit configuration setup shipped with AIX,
/etc/security/audit/config, the BIN mode alternates between /audit/bin1 and
/audit/bin2. When one BIN is full (the binsize parameter determines the size of
the bin), the audit switches to the other BIN file while adding the accumulated
data in the first file to the audit trail (defined in
/etc/security/audit/bincmds), /audit/trail. Use "audit shutdown" to be certain
that all audit records have been added to /audit/trail. The BIN mode audit
record is in binary format. You can read it with audit commands such as
auditpr.
In STREAM mode, the default AIX configuration provides a program to read the
STREAM buffer and processes each record with the commands found in
/etc/security/audit/streamcmds. These commands format the output into
human-readable form and write it in /audit/stream.out. This file is NOT
cumulative; it is restarted every time the audit is restarted. The STREAM audit
trail can be read in real time by reading /audit/stream.out, or by sending
output directly to a terminal or printer.
-------------------------------------------------------------------------------
Starting and Stopping Audit
There are five audit subcommands for invoking auditing. They are as follows:
audit start - to activate the audit subsystem (This is
the only correct way to start audit.)
audit shutdown - to stop auditing subsystem, processing
final BIN records and removing the
/audit/auditb file that is used as an
"active" indicator by the audit modules
audit off - to suspend auditing temporarily
audit on - to resume audit after audit off
audit query - to display the status of auditing
NOTE: Using audit commands in the wrong order can confuse the auditing
subsystem. If the auditing subsystem gets confused, reset everything by
deleting all files in the /audit directory (except trail, stream.out and bin
files).
Auditing can be run at the discretion of the system administrator. Depending on
the environment, it is usually not necessary to have auditing running at all
times. If it is configured to monitor a large number of events or objects at
all times, the amount of data generated would be so substantial that its
overhead would outweigh its benefit. It is worthwhile to take time to configure
auditing to collect selected information.
To start auditing at system startup, add the following line to the /etc/rc
file, just prior to the line reading dspmsg rc.cat 5 'Multi-user initialization
completed':
/usr/sbin/audit start
If auditing is running at all times, make sure to run or to add the following
in the /usr/sbin/shutdown script to properly shut down:
/usr/sbin/audit shutdown
-------------------------------------------------------------------------------
Auditing Configuration
All auditing related configuration files reside in /etc/security/audit. The
/etc/security/audit/config file contains the key audit controls. It has the
following stanzas:
start - specifies whether BIN or STREAM
(or both) should be used for auditing
bin and stream - contain controls for each mode; the names
of the BIN files are specified here
classes - defines several groups (classes) of
auditing events
The predefined classes are: general, objects, SRC, kernel, files, SVIPC, mail,
cron, and TCPIP. New classes can be defined using the auditing events in the
/etc/security/audit/events file. All audit classes except the objects class are
associated with user IDs. For example, audit the events defined as "general"
and "TCPIP" for user root.
user stanza - lists specified users and the audit classes
assigned to them; each username must be the
login name of a system user or the string
'default'
An example of this stanza is as follows:
users:
root = general
joe = general, files, TCPIP
default = general
When auditing starts, it ALWAYS audits the events specified for every user ID
defined in the config file and ALL the objects defined in
/etc/security/audit/objects. If the objects' audit records are not wanted,
remove or comment out (using an *) the objects defined in the objects file.
If there are specific classes of events that are not wanted as audit records,
specify No_Events for that specific class in the config file.
For example:
files = No_Events
or
tcpip = No_Events
The objects file contains all objects to be audited when auditing is active. A
user defined object is displayed as:
/home/joe/my.stuff:
r = "JOE_READ"
w = "JOE_WRITE"
The names JOE_READ and JOE_WRITE are referenced in the /etc/security/events
file to define the format of the auditpr output:
JOE_READ = printf "%s"
JOE_WRITE = printf "%s"
NOTE: There is no need to add the newly added objects to the objects stanza in
the /etc/security/audit/config file, since the objects line is not referenced.
Only the objects file is referenced.
The streamcmds file has commands that are entered for STREAM audit records. The
default file contains one command.
Enter:
/usr/sbin/auditstream | auditpr > /audit/stream.out &
Adding the -v flag for the auditpr command improves this command at the expense
of having more information. Without -v, full path names for files are not shown
in the audit output; only file descriptors are recorded.
To limit the amount of data collected during the auditing operation, use the -c
option on the auditstream command to select a specific class of events as
defined in the config file, or use the auditselect command to select specific
events.
For example:
NOTE: This command must be all on one line in the streamcmds file. This command
will collect only FILE_Open event records.
/usr/sbin/auditstream | /usr/sbin/auditselect -e "event == FILE_Open"
| auditpr -v > /audit/stream.out &
NOTE: The following command will limit data collection to only the TCPIP class
of events as defined in the config file.
/usr/sbin/auditstream -c tcpip | auditpr -v > /audit/stream.out &
The bincmds file contains commands that are entered whenever a BIN file fills
or when auditing is shut down. The file distributed reads like the following:
/usr/sbin/auditcat -p -o $trail $bin
The environment variables in the preceding command are defined while auditing
is active. The auditselect command can be added to select specific events,
reducing the amount of audit records.
The bincmds file will only collect audit records that match USER_SU or
USER_Login audit events. Enter:
/usr/sbin/auditselect -e "event== USER_SU || event== \
USER_Login" $bin > /audit/trail.login
-------------------------------------------------------------------------------
Auditing a User
For example:
1. To audit classes, use the fastpath command smit chuser.
*User NAME [joe]
AUDIT classes [general, files]
A user stanza should be displayed for joe in /etc/security/audit/config
file.
2. At the command line, edit the /etc/security/config file. In the classes
stanza, add the following new class:
procmon = PROC_Create, PROC_Delete, PROC_Execute
In the users stanza, the following could exist:
joe = procmon
The newly assigned audit classes will take effect at the next login for
user joe.
3. The BIN mode audit trail can be read with the following:
auditpr -v < /audit/trail | more
The STREAM mode audit file /audit/stream.out can be viewed directly. Remember
that the /audit/stream.out file is rewritten each time the auditing subsystem
is started. Save the old stream.out before starting auditing.
If you do not want the objects audit records when auditing a user ID, comment
out the objects defined in the /etc/security/audit/objects file or rename this
file.
-------------------------------------------------------------------------------
Auditing an Object
In the following example, all processes writing to the /etc/utmp file will be
audited.
1. Edit the /etc/security/audit/objects file to add the following:
/etc/utmp:
w = "UTMP_WRITE"
2. Edit the /etc/security/audit/events file to include the following:
* /etc/utmp
UTMP_WRITE = printf " %s "
The audit record is displayed as follows:
UTMP_WRITE root OK Wed Jul 12 12:12:25 1995 init
In this case, the init process owned by root wrote to the file.
NOTE: The length of an audit event or object name cannot exceed 15 characters.
This limit is defined in the header file /usr/include/sys/audit.h, ah_event
[16]. The following error message usually indicates an invalid event or object
name.
"auditevents (): Invalid argument"
-------------------------------------------------------------------------------
Disk Space Consideration
Each record in the audit trail takes about 50 to 150 bytes depending on what
mode is used and whether the verbose mode flag is specified. This means that
1MB of data could contain about 6800 entries.
-------------------------------------------------------------------------------
Understanding the Output
It is important to specify what information should be reviewed while auditing.
Although you can configure auditing to record events of interest, there may
still be too much data to be useful when viewed all at once. The auditselect
command can be used with auditpr to sort through volumes of information and
pull out only that which is needed for a specific report. It can be used to
pull all data from a specific time period, for a specific user, or for a
specific event, or any combination of these three.
For example:
/usr/sbin/auditselect -f /audit/pick \
/audit/trail | /usr/sbin/auditpr -v
The /audit/pick file reads as follows:
command == rlogin && \
time >= 08:00:00 && time <= 17:00:00 && \
data >= 04/01/96 && date <= 04/12/96
This command reports the use of the rlogin command within the specific time
interval (8AM-5PM between April 1 and April 12).
The compressed trail data from the binmode auditing is not in the same format
at AIX version 3.2 as it is in 4.1 or later. There is a utility to convert the
data from a pre-AIX Version 4 format to the Version 4 format. It is a command
called auditconv.
-------------------------------------------------------------------------------
Common Problems with Auditing
Errors When Starting Audit
o There are certain errors that appear when running audit start.
Error Message:
** failed setting kernel audit objects
This occurs when there is a syntax error in the /etc/security/audit/objects
file.
o Error Message:
auditbin: ** failed backend command
/etc/auditcat -p -o /audit/trail -r /audit/bin1
This error can be corrected by removing or renaming the BIN files. It is
sometimes helpful to run audit shutdown again and then to retry audit
start.
o Error in config file:
It is necessary to have the "user" stanza in the /etc/security/audit/
configuration file or the following error will display when you start
auditing:
Unable to find the user stanza in /etc/security/audit/config
If it is not obvious that the user stanza is missing, verify that each of the
classes are defined on a single continuous line.
-------------------------------------------------------------------------------
Data Overload
Given the way that cron and the TCPIP code is written, each sets up its own set
of audit events. These events will get written into the audit trail regardless
of how the config files are set up. The workaround is to use auditselect to
exclude these events when generating the audit report. TCPIP sessions, ftpd,
rexecd, and telnetd all call auditproc() to set up process auditing using the
class tcpip in /etc/security/audit/config. The same thing is done in the cron
code (at, cron, and cronadm) for the cron class in /etc/security/audit/config.
These events will be written into the audit trail. The best thing to do is to
filter them using auditselect.
For example:
auditselect -e"event!=AT_JobAdd && event!=AT_JobRemove && ..."
This will exclude events AT_JobAdd and At_JobRemove and so on.
Or select on the command name:
auditselect -e"command!=cron && command!=at && ..."
This excludes recording related to commands, at, cron, and so on.
-------------------------------------------------------------------------------
Fixes for Auditing Subsystem
As defects arise, updates to the AIX audit code are made.
Techdocs Ref:90605195614808 4FAX Ref:4824
Setup of System Accounting in AIX 3.2 [acct.setup.32.cmd]
Setup of System Accounting in AIX 3.2
-------------------------------------------------------------------------------
Contents
About This Document
Related Documentation
What to Install to Get Accounting Working on a System
Steps for Modifying the Root User Profile
Steps for Setting Up System Accounting
Fields of the Daily Usage Report
Fixes and Problems
Diagnosing Problems
-------------------------------------------------------------------------------
About This Document
This document describes the accounting setup procedures for AIX Version 3.2.
The setup of system accounting as documented in the 3.2 AIX InfoExplorer has
some errors. This document describes the Version 3.2 setup procedures and
provides the corrections to enable system accounting under AIX version 3.2.
Access to the InfoExplorer and system documentation for AIX Version 3.2 require
the setup person to be root. In this document, root user will be used for
setting up system accounting and the user adm for running the reports.
Modify the root user .profile to provide access to the required executables.
Modification of the root profile is optional, but will probably help with
administration.
Related Documentation
InfoExplorer
-------------------------------------------------------------------------------
What to Install to Get Accounting Working on a System
Make sure the following product is installed:
bosext2.acct.obj - 'lslpp -l bosext2.acct.obj'
This command should report the state as COMMITTED or APPLIED if the fileset is
installed. To get the latest level accounting software, refer to the section
"Fixes and Problems" later in this document.
-------------------------------------------------------------------------------
Steps for Modifying the Root User Profile
1. Log in as root.
2. Edit the .profile file with your favorite text editor.
3. Locate the PATH statement in the .profile file. It must include the
/usr/sbin/acct and /var/adm/acct directories as shown in the following
example:
PATH= /usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:
/usr/sbin/acct:/var/adm/acct ; export PATH
4. Locate the export statement in the .profile. If the PATH statement given
above was not used, the PATH variable must be included.
-------------------------------------------------------------------------------
Steps for Setting Up System Accounting
1. Log in to the system as root user. As described previously, root is the
accounting administrator.
2. Enter:
su - adm -c /usr/lib/acct/nulladm /var/adm/wtmp /var/adm/pacct
This command ensures correct permissions and provides access to the wtmp
and pacct files.
3. Update the /etc/acct/holidays file for the current year using a favorite
text editor. The following notes will help clarify the format of this
file:
o An asterisk (*) in column 1 denotes a comment.
o Define prime time on the first data line (the first line that is not a
comment) using a 24-hour clock. The line will consist of three 4-digit
fields, in the following order:
- current year
- when (hhmm) prime time begins
- when (hhmm) prime time ends
Leading blanks are ignored. Midnight can be entered as either 0000 or
2400.
For example, to specify the year 1992 with prime time beginning at
8:00 a.m. and ending at 5:00 p.m., specify:
1992 0800 1700
o Define the company holidays for the year on the next data line. Each
line contains four fields in the following order:
- day of the year
- month
- day of the month
- description of the holiday
The day-of-the-year field contains the number of the day of the year
in which the holiday falls. It must be a number from 1 through 365
(366 on a leap year like 1992).
4. The file systems that will be included for accounting information need to
be configured in the /etc/filesystems file. For each file system that will
be included, add the following information to its stanza:
account = true
5. Indicate the data file that will be used for printer data by adding the
following line to the /etc/qconfig stanza for that printer:
acctfile = /var/adm/qacct
This entry must be added to the queue stanza and not the device stanza. If
the entry is added by editing the /etc/qconfig file, enter the following
command to re-digest the qconfig file:
enq -d
Another method of adding this entry to the qconfig file would be to enter
the command smitty chque. Then follow the next steps:
1. Select queue name.
2. Enter /var/adm/qacct for the ACCOUNTING FILE pathname.
These steps will update the appropriate stanza in /etc/qconfig file and
will re-digest the qconfig file.
NOTE: The printer accounting will not record usage for network printers,
transparent printers, or PostScript printers. In addition, all print
queues must use the same accounting file:
/var/adm/qacct
The pac command (printer accounting command) is more versatile because it
allows separate accounting files for each printer. For more information,
see InfoExplorer.
6. Create directories /var/adm/acct/nite, /var/adm/acct/fiscal, and
/var/adm/acct/sum if they do not already exist. These directories should
be owned by the user adm with a group ID of "adm". Use the following
series of commands to create these directories:
su - adm
cd /var/adm/acct
mkdir nite fiscal sum
chown adm:adm nite fiscal sum
chmod 755 nite fiscal sum
exit
7. Modify the crontabs file for the adm user for automated accumulation of
accounting data. The crontabs file for the adm user is in
/var/spool/cron/crontabs/adm.
To edit the adm crontab, log on as root and enter:
su - adm -c crontab -e
This will bring up the adm crontabfile in a vi session. You may need to
uncomment or add some entries, and times may need to be modified to suit
the installation's time requirements. An example of the entries needed is
as follows:
10 23 * * 0-6 /usr/lib/acct/runacct 2> \
/usr/adm/acct/nite/accterr > /dev/null
0 23 * * 0-6 /usr/lib/acct/dodisk > /dev/null 2>&1
0 * * * * /usr/lib/acct/ckpacct > /dev/null 2>&1
15 4 1 * * /usr/lib/acct/monacct > /dev/null 2>&1
o The first line starts runacct at 11:10 pm (10 23), each Sunday through
Saturday (0-6). runacct runs the daily reporting.
o The second line starts disk accounting at 11:00 pm (0 23), each Sunday
through Saturday (0-6), before starting runacct.
o The third line checks the /var/adm/pacct file to ensure that it does
not get too large at 0 minutes past each hour (0 *) every day (*).
If the free space in /var falls below 500 blocks, then ckpacct turns
off accounting until space is made available. A loss of accounting
data will result during the period that accounting is turned off.
ckpacct will turn accounting on again when more space is available.
THERE IS NO NOTIFICATION unless the MAILCOM variable is set as
follows:
MAILCOM="mail root adm"
The variable can be set in the ckpacct and runacct scripts or in the
/etc/environment file. If MAILCOM is set in both places the setting in
ckpacct and runacct will be used.
o The fourth line runs the monthly accounting reports at 4:15 am on the
first of every month.
8. Turn on process accounting by adding the following line to the /etc/rc
file.
/usr/bin/su - root -c /usr/sbin/acct/startup
The startup command records the time that accounting was turned on and
cleans up the previous day's accounting files.
If the system is not going to be rebooted at this time, run the preceding
startup command from the root command line to start process accounting.
-------------------------------------------------------------------------------
Fields of the Daily Usage Report
The meaning of the fields in the accounting Daily Usage Report are as follows:
UID
User ID number
LOGIN NAME
Login name of user
CPU PRIME
Cumulative CPU minutes during prime hours
CPU NPRIME
During non-prime hours
KCORE PRIME
Cumulative minutes spent in the kernel during prime hours
KCORE NPRIME
During non-prime hours
BLKIO PRIME
Cumulative blocks transferred during prime hours
BLKIO NPRIME
During non-prime hours
RW/WR PRIME
Cumulative blocks read/written during prime hours
RW/WR NPRIME
During non-prime hours
CONNECT PRIME
Cumulative connect time (minutes) during prime hours
CONNECT NPRIME
During non-prime hours
DISK BLOCKS
Cumulative disk usage
PRINT
Queuing system charges (pages)
FEES
Fee for special services
# OF PROCS
Count of processes
# OF SESS
Count of login sessions
# OF SAMPLES
Count of count of disk samples
-------------------------------------------------------------------------------
Fixes and Problems
Fixes for AIX Version 3.2.5 and later can be downloaded via the Internet with
the FixDist service.
On very large systems, if there are over 1000 users in the /etc/passwd file,
the /usr/sbin/acct/dodisk script must be changed. At line 136 in dodisk, the
diskusg command must be changed to read as follows:
diskusg -Unnnnn $args > dtmp
nnnnn is the new maximum number of users for which disk accounting should be
done. (When the maximum number is too small, the dodisk script will return an
error message that tells the user to use -uxxxx. It should say -Uxxxx.)
On AIX 3.2.5 and Greater
o IX38748 corrects the problem of a user missing the output from individual
users for the CPU and DASD output.
o IX39408 corrects the problem of process accounting failing on a diskless
workstation.
o IX42292 corrects the problem of the runacct account command truncating the
wtmp file during its nightly run, causing connection accounting to be lost
for users who do not log off.
o IX40232 corrects the problem of daily and monthly per-user memory
accounting being off by a factor of exactly 200.
o IX42322 corrects the problem of a 64MB limit on system process accounting.
With this problem, any process that used more than 64MB of memory caused an
overflow, and the process size reported by the ac_mem field was inaccurate.
o IX43161 corrects the problem of acctprc2 core dumping with more than 500
users.
o IX41228 corrects the problem of acctdisk entering into an infinite loop if
the input file is corrupted.
o IX40305 corrects the problem of the /var/adm/acct/sum/login log file not
being updated in 3.2.5.
-------------------------------------------------------------------------------
Diagnosing Problems
General Information Needed
o Which accounting command is being used?
acctcms acctmerg
acctcom acctprc
acctcon acctprcl
acctdisk acctprc2
All of these accounting commands accept input from standard input (as in
acctcms < /usr/adm/pacct) and redirect to standard output (as in acctcmd <
/usr/adm/pacct > /tmp/report). Find out which accounting file is being used
as standard input and where the output is being directed by entering:
acct_cmd < In_file > Out_file
o Defaults are standard in and standard out. Exact syntax is very important.
o How is accounting started, via cron or via command line?
If accounting is started via cron, note the crontab file entries. If
accounting is run from root, then the entry is in
/usr/spool/cron/crontabs/root. If accounting is run from adm, then the
entry is in /usr/spool/cron/crontabs/adm.
When Accounting Fails
1. Check to see what state accounting is in. Look at the
/usr/adm/acct/nite/active file. States are as follows:
setup
wtmpfix
connect1
connect2
process
merge
fees
disk
queueacct
mergetacct
cms
userexit
cleanup
2. Check accterr for additional messages.
3. Refer to section "How To Fix General Accounting Problems" in InfoExplorer.
4. Restart runaccount.
Other Items to Check
o Is the /usr directory out of space?
o Does wtmp have records with inconsistent data stamps?
o Is cron failing? (Check mail for root and adm.)
Techdocs Ref:90605194214786 4FAX Ref:2486