[ Previous | Next | Contents | Home | Search ]
AIX Version 4.3 Understanding the Diagnostic Subsystem for AIX

Debugging Hints for Diagnostic Kernel Extension


Starting Trace for Diagnostic Kernel Extension

The Diagnostic Controller will load the Kernel Extension(s) for each device that requires it. This is specified by the PDiagRes->KernExt ODM stanza for the device. If using DIAGEX or PDIAGEX, there is a trace hook built in for debugging purposes.

To use this trace hook, you first must make sure that the trace command is installed. This command is part of the bos.sysmgt.trace fileset.

To run trace, perform the following:

trace -j 355		// Invoke trace
> trcon			// Start trace
> !diag -d "device_name"// Run diagnostics against the device 
> trcoff		// Stop trace
> quit			// Quit

To generate a trace file, perform the following:

trcrpt -o /tmp/diagex.trc	

This trace file will contain all the steps performed by the diagnostic kernel extension. To understand the tags, you must use the diagex source code. This source code can be found at src/bos/kernext/diagex for DIAGEX and src/bos/kernext/pdiagex for PDIAGEX.


Looking at an Illegal Trap

In some instances, an Illegal Trap Instruction may occur if some application unloads their SLIH or kernel extension, without having previously unpinned its memory. This can also happen if the Diagnostic Kernel Extension 'close' routine is not called on exit.>

When this happens, a screen similar to the following may appear if the debugger is enabled. The sign that some application did not unpin some code before unloading is that the dump will show ff_free.

The address passed to ff_free is in (r29) or r30. (??) Use the (s)creen command to trace back until you see a familiar function name. In the following example, the SLIH mps_interrupt was indicted.

------------
Trap Occurs
------------
GPR0  00000000 2FF3B188 00192DF0 00000016 007FFFFF C0000000 00009030 2FF3B400   
GPR8  00000000 00000000 00000000 00000010 0014032C DEADBEEF DEADBEEF DEADBEEF   
GPR16 DEADBEEF DEADBEEF 200004B0 DEADBEEF DEADBEEF DEADBEEF 2FF3B2C0 00000000   
GPR24 00000000 00161BF8 C0000420 03762428 0015FF40 01A1C5A0 01A1C5A8 0015FF40   
                                                                                
MSR 00029030  CR   44224828  LR   0014032C  CTR   000908A8  MQ   00000000       
XER 00000000  SRR0 00140334  SRR1 00029030  DSISR 40000000  DAR  00000000       
                                                                                
IAR 00140334  (ORG+00140334)  ORG=00000000   Mode: VIRTUAL                      
00140330   5400D97E 0C800000 387F0000 4BECADC5   |T..~....8...K...|             
                    |    tweqi   r0,0x0                                         
00140340   81810058 30210050 7D8803A6 BBA1FFF4   |...X0!.P}.......|             
                                                                                
                    |                                                           
00140330   5400D97E 0C800000 387F0000 4BECADC5   |T..~....8...K...|             
00140340   81810058 30210050 7D8803A6 BBA1FFF4   |...X0!.P}.......|             
00140350   4E800020 00000000 00002041 80030100   |N.. ...... A....|             
00140360   00000000 00000174 00076666 5F667265   |.......t..ff_fre|             
00140370   65000000 80E20328 BF81FFF0 7C0802A6   |e......(....|...|             
00140380   2C070000 90010008 9421FFB0 3B830000   |,........!..;...|             
00140390   41820050 80E201E8 38640000 83810040   |A..P....8d.....@|             

Illegal Trap Instruction Interrupt in Kernel                              
 
>0>

---------------------------------------
Use (s)creen to display contents of R29
---------------------------------------
>0> s 1A1C5a0 100                                                               
GPR0  00000000 2FF3B188 00192DF0 00000016 007FFFFF C0000000 00009030 2FF3B400   
GPR8  00000000 00000000 00000000 00000010 0014032C DEADBEEF DEADBEEF DEADBEEF   
GPR16 DEADBEEF DEADBEEF 200004B0 DEADBEEF DEADBEEF DEADBEEF 2FF3B2C0 00000000   
GPR24 00000000 00161BF8 C0000420 03762428 0015FF40 01A1C5A0 01A1C5A8 0015FF40   
                                                                                
MSR 00029030  CR   44224828  LR   0014032C  CTR   000908A8  MQ   00000000       
XER 00000000  SRR0 00140334  SRR1 00029030  DSISR 40000000  DAR  00000000       
                                                                                
IAR 00140334  (ORG+00140334)  ORG=00000000   Mode: VIRTUAL                      
00140330   5400D97E 0C800000 387F0000 4BECADC5   |T..~....8...K...|             
                    |    tweqi   r0,0x0                                         
00140340   81810058 30210050 7D8803A6 BBA1FFF4   |...X0!.P}.......|             
                                                                                
           |                                                                    
01A1C5A0   01A29850 0000A518 01DF0004 325E9F94   |...P........2^..|             
01A1C5B0   00000000 00000000 00481007 010B0001   |.........H......|             
01A1C5C0   00000BF0 0000010C 00000000 000000E4   |................|             
01A1C5D0   00000000 00000000 000000F0 00020001   |................|             
01A1C5E0   00020002 00040003 00020003 314C0000   |............1L..|             
01A1C5F0   00000000 00000000 00000000 00000000   |................|             
01A1C600   00000000 2E746578 74000000 00000000   |.....text.......|             
                                                                                

------------------------------------------
Press enter until you find a function name
------------------------------------------
>0> <enter a bunch of times>                                                                
GPR0  00000000 2FF3B188 00192DF0 00000016 007FFFFF C0000000 00009030 2FF3B400   
GPR8  00000000 00000000 00000000 00000010 0014032C DEADBEEF DEADBEEF DEADBEEF   
GPR16 DEADBEEF DEADBEEF 200004B0 DEADBEEF DEADBEEF DEADBEEF 2FF3B2C0 00000000   
GPR24 00000000 00161BF8 C0000420 03762428 0015FF40 01A1C5A0 01A1C5A8 0015FF40   
                                                                                
MSR 00029030  CR   44224828  LR   0014032C  CTR   000908A8  MQ   00000000       
XER 00000000  SRR0 00140334  SRR1 00029030  DSISR 40000000  DAR  00000000       
                                                                                
IAR 00140334  (ORG+00140334)  ORG=00000000   Mode: VIRTUAL                      
00140330   5400D97E 0C800000 387F0000 4BECADC5   |T..~....8...K...|             
                    |    tweqi   r0,0x0                                         
00140340   81810058 30210050 7D8803A6 BBA1FFF4   |...X0!.P}.......|             
                                                                                
           |                                                                    
01A1CDF0   41820010 306300CC 48000479 80410014   |A...0c..H..y.A..|             
01A1CE00   38600000 4800000C 3860FFFF 48000004   |8`..H...8`..H...|             
01A1CE10   80010088 7C0803A6 30210080 BBC1FFF8   |....|...0!......|             
01A1CE20   4E800020 00000000 00002041 80020201   |N.. ...... A....|             
01A1CE30   00000000 00000780 000D6D70 735F696E   |..........mps_in|             
01A1CE40   74657272 75707400 00000000 BDA1FFB4   |terrupt.........|             
01A1CE50   80A20004 39C30000 80650060 7C0802A6   |....9....e.`|...|             
                                                                                

[ Previous | Next | Contents | Home | Search ]