07/18/96, 4FAX# 5624 SNA Testcases: Tracing and Submitting SPECIAL NOTICES Information in this document is correct to the best of our knowledge at the time of this writing. Please send feedback by fax to "AIXServ Information" at (512) 823-4009. Please use this information with care. IBM will not be responsible for damages of any kind resulting from its use. The use of this information is the sole responsibility of the customer and depends on the customer's ability to eval- uate and integrate this information into the customer's operational environment. +----------------------------------------------------------+ | | | NOTE: The information in this document has NOT been | | verified for AIX 4.1. | | | +----------------------------------------------------------+ ABOUT THIS DOCUMENT This document is intended as a guide for the collection and submission of traces and logs for HCON, DLC, and AIXSNA problem determination (PD). Three major sections are: 1. The AIXSNA Testcase 2. More About Tracing 3. Submitting The Testcase THE AIXSNA TESTCASE In general, an AIXSNA testcase (TC) should be created as outlined here; however if requests for specific traces or logs have been made, that output may have to be moved to /var/sna for testcase collection. Call 1-800-call aix if you have questions. 1. Stop AIXSNA if possible. # sna -stop sna -type cancel If this is difficult at your site due to resource avail- ability requirements or if it would mask the problem, then leave active. 2. Clean out all files from /var/sna. # cd /var/sna # rm * SNA Testcases: Tracing and Submitting 1 07/18/96, 4FAX# 5624 Please move or remove all old files. AIXSNA will recreate files as required. 3. Create the problem. Activate all requested traces and logs (see the section "More About Tracing). Reproduce the problem as quickly as possible. Stop all traces and logs as soon as the problem has occurred. The goal is to create the smallest testcase that will allow complete problem anal- ysis. 4. Describe the problem. # vi /var/sna/readme.problem Create a file that describes the problem plus any obser- vations about resource states on both AIX and the other platform that AIXSNA is (attempting to) communicating with. Please note the time when the problem occurred (AIX date command or system clock). 5. Collect the testcase. # cd /var/sna # /usr/lpp/sna/bin/getsnapd All the files in /var/sna plus system logs and traces are placed in pd.tar.Z by the getsnapd script. Send this file as a binary image via one of the methods below (see Submitting The Testcase). Please change the file name to .tar.Z where is the PMR number or xmenu item number opened to report your problem. NOTE: PLEASE DO NOT SEND A TESTCASE UNLESS YOU HAVE AN ACTIVE ITEM. MORE ABOUT TRACING Traces are often the key to problem determination. Dif- ferent types of traces and logs capture different types of problem determination information. This section explains the use of some of the most important traces. NOTE: Start all traces and logs before you recreate the problem and stop them all immediately after recreating the problem. AIXSNA Link Station (LS) Trace: The link station trace is the most important trace for AIXSNA Problem Determination. A link station trace can be started in the link station profile, from the command line, or from SMIT. The AIXSNA LS trace captures line flow infor- mation; this type of information can also be captured in VTAM BUFFER, CCW, or LINE traces or via a "line scope" such as a Sniffer or IBM DataGlance. From the Link Station Profile: SNA Testcases: Tracing and Submitting 2 07/18/96, 4FAX# 5624 Set the parameter 'Trace Link?' to YES. Set the parameter 'Trace Size' to LONG. Set ALL of the Restart parameters to NO. This will start a link station trace every time the link station is started. This is a good way of tracing link activation problems. The trace will be given the same name as the link station and will be placed in the /var/sna directory. Since the trace is started every time the link station is started, previous instances of the trace file will be overwritten unless they are renamed or moved to another directory. From the Command Line: # sna -trace on -l to start a LS trace. # sna -trace off -l to stop a LS trace. The LS trace can be started just before reproducing the problem and stopped just after. Set 'Trace Link = no' and all restart parameters to 'no' in this LS profile to avoid the LS trace being overwritten when the LS restarts. From SMIT: smit sna Problem Determination Aids Control SNA Traces Start and Stop Selected SNA Traces specify the Link Station name and hit enter. The AIXSNA LS trace either traces everything or nothing; tracing can not be limited to a specific session or LU. AIXSNA LS traces can be formatted with the AIX trcrpt command; however a more useful formatter is provided with AIXSNA. The following command will result in three ASCII output files; a summary (-s) file named .sum, a detailed (-d) file named .det, and a file showing the data in hex format named .hex. # snaformat -l -s -d -h If you want to see physical link level flows (XID, SNRM, RR, etc.) include the (-p) flag. To format only the flows for a specific LU use "-i iod="; for example to see summary and hex reports for all the flows between OAF and DAF 01 and 07 (or LFSID's involving 07): # snaformat -l -s -h -i iod=00007 iod=00107 AIXSNA LOGS AIXSNA logs can be controlled from the command line or from SMIT. For problem determination on a specific problem it is often simpler to use the command line. Anything that works from the command line can also be executed from a shell script. For continuous logging of a specific type of infor- mation it is often simpler to use SMIT. SNA Testcases: Tracing and Submitting 3 07/18/96, 4FAX# 5624 AIXSNA Logging Impact: AIXSNA has a minimal default level logging which can not be terminated. The following will invoke the minimal trace and log impact on a given system. # trcstop # sna -trace off # sna -setlogs -e -zs 200000 -zi 0 -zf 0 -w 2 It is suggested that most customers invoke additional logging for normal operations; this suggested level has little impact on processing and will provide useful informa- tion if an AIXSNA problem is reported. # sna -setlogs -e -zs 1500000 -zi 0 -zf 0 -w 5 # sna -trace only -e ras opint For problem determination efforts it is often simpler if all AIXSNA traces and logs are directed to one log file and the "system log" (-s) is used for the SNA service log. LS traces can not be directed to the SNA service log file, /var/sna/snaservice.[n]. API traces started with the sna command will be directed to snaservice log when the "system log" (-s), rather than the "SNA log" (-e), is active. Since the "system log" uses AIX trace channel zero you can not start additional instances of the AIX trace (i.e. any trace called will fail with a 0454-072 message). AIXSNA API traces can not be invoked via the sna command unless the "system log" (-s) is active. An API traces can be invoked via the AIX trace command with the appropriate hookid's when the "SNA log" (-e) is active. AIX trace command output goes to /var/adm/ras/trcfile. # trace -aj 271,390 o To stop an AIX trace: # trcstop o To list all AIX trace hooks: # trcrpt -j AIXSNA HOOKID'S: 271 AIXSNA API 390 AIXSNA CPI-C 27A AIXSNA events 27B AIXSNA errors 325 MPTN: "AnyNet" 281 SNA LU0 or GSNA API trace 282 SNA Network Management API DLC HOOKID'S: SNA Testcases: Tracing and Submitting 4 07/18/96, 4FAX# 5624 1CD Ethernet Driver 1CE tokdd Token Ring Driver 1CF C327DD 3270 Connection Adapter DD 224 MPQPDD MPQP Driver 225 X25DD X.25 Driver 227 X.25 Performance 22C MPADD Single Port Multiprotocol Adapter 245 SDLC Driver Performance 246 SDLC Driver monitor 263 FDDIdd FDDI Driver 280 HIADD SYSMSLA - HCON over DFT 350 CATDD ESCON SNA Channel 358 BLKMUXDD Block MUX SNA Channel AIXSNA API TRACES: If your problem is API related this trace should be col- lected at the same time as a LS trace or a pc_trace event trace. It is also helpful if you include a description of the application (or even a small code fragment) with the problem description in /var/sna/readme.problem file. Also please identify the application by product name and vendor. Many applications include logging facilities which may be helpful in API level problem determination. # sna -trace on -a on // AIXSNA API trace // # sna -trace on -c on // AIXSNA CPI-C trace // # sna -trace on -g on // AIXSNA GSNA API trace // AIXSNA ANYNET TRACE: If you have an AnyNet problem use the following. # sna -trace on -anynet short AIXSNA EVENT TRACE: This trace can be configured to collect information on many types of AIXSNA internal events. Detailed help is available via the F1 key in SMIT. # smit sna Problem Determination Aids Control SNA Traces Start and Stop Selected SNA Traces Event trace (list events) The event trace can be very helpful in problem determi- nation, however most events will rapidly fill and wrap the sna service log. The following events will have little impact on processing and will provide useful information if an AIXSNA problem is reported. # sna -trace only -e ras opint Other events should only be logged while attempting problem determination on a specific AIXSNA problem. As an example, the following traces would be helpful for a CPI-C applica- tion level problem. SNA Testcases: Tracing and Submitting 5 07/18/96, 4FAX# 5624 # sna -setlogs -s -zs 5000000 -w 5 # sna -trace on -l your_LS -a on -c on -e 'ras opint pc_trace' AIXSNA FLOW TRACE: The flow trace logs entry and exit of AIXSNA functions. Customer should only activate this trace if requested by IBM AIXSNA support. AIXSNA COMPONENT TRACE: The component trace limits AIXSNA internal tracing to spe- cific AIXSNA components (daemons). Customer should only activate this trace if requested by IBM AIXSNA support. AIXSNA MBUF TRACE: Use this trace if you believe you have a problem with mbuf's caused by AIXSNA. Please read both the AIX tuning and AIXSNA tuning faxes and review the buffer pool tuning infor- mation before collecting an AIXSNA mbuf trace. Titles are: o Network Tuning (call 1-800-call-aix request document "network.tuning.tcp".) o SNA Services/6000 Tuning and Performance Tips (Call 1-800-4fax, requests document # 1529.) # sna -trace on -mbuf on AIX SYSTEM LOG The AIX system log is often useful in AIXSNA problem deter- mination and is collected by the getsnapd script. You can format the system log with the errpt command. A short format shows one line per logged event. # errpt ERROR_ID TIMESTAMP T CL RESOURCE_NAME ERROR_DESCRIPTION E9376250 0429095696 P S lu0server CONFIGURATION E9376250 0429095596 P S lu0server CONFIGURATION 1581762B 0426012696 T H hdisk1 DISK ERROR C14C511C 0426012596 T H scsi0 ADAPTER ERROR A detailed format can then be collected for only the events after a specific timestamp displayed in the short format. For example if we are only interested in the lu0server events: # errpt -s 0429095596 -a The system error log should be periodically reviewed and cleared to keep the log to a reasonable size. # errclear 0 See InfoExplorer for details. SNA Testcases: Tracing and Submitting 6 07/18/96, 4FAX# 5624 AIX SCRIPT COMMAND The AIX script command is useful if you want to capture all I/O for standard input, standard output, and standard error as descriptive information. Recording is stopped by issuing from the command line. For example to report a problem with the verifysna command: # script /var/sna/readme.problem # verifysna -R # # echo y |/usr/lpp/sna/bin/getsnapd See InfoExplorer for details. COLLECTING A TESTCASE AIXSNA includes a shell script that collects useful AIXSNA Problem determination information and AIX system informa- tion. All files in /var/sna are collected so please remove all files from /var/sna prior to recreate the problem. # /usr/lpp/sna/getsnapd Answer "yes" to the "stop traces" question. The script may take several minutes to complete. All information will be in the /var/sna/pd.tar.Z file. Please send file pd.tar.Z as a binary image named .tar.Z. For example if you have PMR 1X234 B567 open for your problem, send the file as 1x234b567.tar.Z. DOCUMENTATION Most aixsna commands output "help" if issued without requi- site arguments. To view "help" issue the following: # sna -trace # sna -setlogs # snaformat Details for sna commands and profile fields are available in smit help (F1) or in AIXSNA manuals: SC31-8211 SNA Server for AIX: User's Guide SC31-8213 SNA Server for AIX: Configuration Reference SC31-8214 SNA Server for AIX: Commands Reference SC31-8215 SNA Server for AIX: Diagnosis Guide and Message AIX commands are documented in InfoExplorer. Most AIX SNA manuals are available as both hardcopy and softcopy. AIXSNA manuals use a DynaText browser that is not compatible with InfoExplorer. A CD with the browser and softcopy manuals is distributed with all new AIXSNA licenses. Softcopy manuals and the browser are also avail- able on the Web at http://www.raleigh.ibm.com/asf/asfprod.html SNA Testcases: Tracing and Submitting 7 07/18/96, 4FAX# 5624 SUBMITTING THE TESTCASE Several methods of submitting testcases are documented in the FAX "AIX Support Family -- Sending Testcases", which is available from SupportLine or from 1-800-IBM-4FAX, document number 1762. SNA Testcases: Tracing and Submitting 8 07/18/96, 4FAX# 5624 READER'S COMMENTS Please fax this form to (512) 823-4009, attention "AIXServ Informa- tion". You may also e-mail comments to: elizabet@austin.ibm.com. These comments should include the same customer information requested below. Use this form to tell us what you think about this document. If you have found errors in it, or if you want to express your opinion about it (such as organization, subject matter, appearance) or make sug- gestions for improvement, this is the form to use. If you need technical assistance, contact your local branch office, point of sale, or 1-800-CALL-AIX (for information about support offer- ings). These services may be billable. Faxes on a variety of sub- jects may be ordered free of charge from 1-800-IBM-4FAX. Outside the U.S. call 415-855-4329 using a fax machine phone. When you send comments to IBM, you grant IBM a nonexclusive right to use or distribute your comments in any way it believes appropriate without incurring any obligation to you. NOTE: If you have a problem report or item number, supplying that number may help us determine why a procedure did or did not work in your specific situation. Problem Report or Item #: Branch Office or Customer #: Be sure to print your name and fax number below if you would like a reply: Name: Fax Number: ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ END OF DOCUMENT (testcase.sna, 4FAX# 5624) SNA Testcases: Tracing and Submitting 9