This section includes some hints and tips that may assist you when you encounter a problem. We recommend that you set up logging from the start. Logs are very useful in determining what is going on with the filters and tunnels. (See Logging Facilities for detailed log information.)
| Error: | Issuing mktun command results in the following error:
insert_tun_man4(): write failed : The requested resource is busy. Problem: The tunnel you requested to activate is already active or you have colliding SPI values. To fix: Issue the rmtun command to deactivate, then issue the mktun command to activate. Check to see if the SPI values for the failing tunnel match any other active tunnel. Each tunnel should have its own unique SPI values. |
| Error: | Issuing mktun command results in the following error:
Device ipsec_v4 is in Defined status.
Tunnel activation for IP Version 4 not performed. Problem: You have not made the IP Security device available. To fix: Issue the following command:
mkdev -l ipsec -t 4 You may have to change -t option to 6 if you are getting the same error for Version 6 tunnel activation. The devices must be in available state. To check the IP Security device state, issue the following command: lsdev -Cc ipsec |
| Error: | Issuing a gentun command results in the following error:
Invalid Source IP address Problem: You have not entered a valid IP address for the source address. To fix: For IP Version 4 tunnels, check to see that you have entered an available IP Version 4 address for the local machine. You cannot use host names for the source when generating tunnels, you may only use host names for the destination. For IP Version 6 tunnels, check to see that you entered an available IP Version 6 address. If you type netstat -in and no IP Version 6 addresses exist, run /usr/sbin/autoconf6 (interface) for a link local auto-generated address (using MAC address) or use ifconfig to manually assign an address. |
| Error: | Issuing a gentun command results in the following error:
Invalid Source IP address Problem: You have not entered a valid IP address for the source address. To fix: For IP Version 4 tunnels, check to see that you have entered an available IP Version 4 address for the local machine. You cannot use host names for the source when generating tunnels, you may only use host names for the destination. For IP Version 6 tunnels, check to see that you entered an available IP Version 6 address. If you type netstat -in and no IP Version 6 addresses exist, run /usr/sbin/autoconf6 (interface) for a link local auto-generated address (using MAC address) or use ifconfig to manually assign an address. |
| Error: | Issuing mktun command results in the following error:
insert_tun_man4(): write failed : A system call received a parameter that is not valid. Problem: Tunnel generation occurred with invalid ESP and AH combination or without the use of the new header format when necessary.
To fix: Check to see what authentication algorithms are in use by the particular tunnel in question. Remember that the HMAC_MD5 and HMAC_SHA algorithms require the new header format. The new header format can be changed using the SMIT fast path ips4_basic or the -z parameter with the chtun command. Also remember that DES_CBC_4 cannot be used with the new header format. |
| Error: | Starting IP Security from Web-based System Manager results in a
Failure message.
Problem: The IP Security daemons are not running. To fix: View which daemons are running by entering the ps -ef command. The following daemons are associated with IP Security:
The cpsd daemon is active only if the digital certificate code is installed (the fileset named gskit.rte or gskkm.rte) and you have configured the IBM Key Manager tool to contain digital certificates. If the daemons are not active, stop IP Security using Web-based System Manager and then restart it, which automatically starts the appropriate daemons. |
| Error: | Trying to use IP Security results in the following error:
The installed bos.crypto is back level and must be updated. Problem: The bos.net.ipsec.* files have been updated to a newer version, but the corresponding bos.crypto.* files have not. To fix: Update the bos.crypto.* files to the version that corresponds with the updated bos.net.ipsec.* files. |
The following sections described errors that can occur when using IKE tunnels.
The IKE tunnels are set up by the communication of the ike
command or the Web-based System Manager VPN panels with three daemons:
| tmd | The Tunnel Manager daemon |
| isakmpd | The IKE daemon |
| cpsd | The certificate proxy daemon |
For IKE tunnels to be properly set up, the tmd and isakmpd daemons must be running. If IP Security is set to start at reboot, these daemons start automatically. Otherwise, they must be started using Web-based System Manager.
The Tunnel Manager gives requests to isakmpd to start a tunnel. If the tunnel already exists or is not valid (for instance, has an invalid remote address), it reports an error. If negotiation has started, it may take some time, depending on network latency, for the negotiation to complete. The ike cmd=list command can list the state of the tunnel to determine if the negotiation was successful. Also, the Tunnel Manager logs events to syslog to the levels of debug, event, and information, which can be used to monitor the progress of the negotiation.
The sequence is:
When the machine is acting as a responder, the isakmpd daemon notifies the Tunnel Manager tmd daemon that a tunnel has been negotiated successfully and a new tunnel is inserted into the kernel. In such cases, the process starts with step 3 and continues until step 7, without the tmd daemon issuing connection requests.
The isakmpd, tmd and cpsd daemons log events to syslog. For isakmpd, logging is enabled using the ike cmd=log command. The /etc/isakmpd.conf configuration file can be set up to specify the logging level. The level can be set to none, errors, isakmp_events, or information.
Note: In versions earlier than AIX 5.1, isakmpd logged to a separate file, which was also specified in /etc/isakmpd.conf.
The configuration file parameter that can be set for logging is log_level. The IKE daemons use the following levels of logging:
The syntax for this option is simply:
log_level
The isakmpd daemon code either initiates by sending a proposal or responds by evaluating a proposal. If the proposal is accepted, a security association is created and the tunnel is set up. If the proposal is not accepted or the connection times out before the negotiation completes, isakmpd indicates an error. The entries in syslog from tmd indicate whether the negotiation succeeded. A failure caused by an invalid certificate logs to syslog. To find out the exact cause of a failed negotiation, check the logging file specified in /etc/syslog.conf.
The syslog facility adds a prefix to each log line, noting the date and time, the machine, and the program. The following example uses googly as the machine name and isakmpd as the program name:
Nov 20 09:53:50 googly isakmpd: ISAKMP_MSG_HEADER Nov 20 09:53:50 googly isakmpd: Icookie : 0xef06a77488f25315, Rcookie :0x0000000000000000 Nov 20 09:53:51 googly isakmpd: Next Payload : 1(SA), Maj Ver : 1, Min Ver : 0 Nov 20 09:53:51 googly isakmpd: Xchg Type : 2 (ID protected), Flag= 0, Encr : No,COMMIT : No Nov 20 09:53:51 googly isakmpd: Msg ID : 0x00000000
To improve clarity, the grep command can be used to extract log lines of interest (such as all isakmpd logging) and the cut command can be used to remove the prefix from each line. The isakmpd log examples in the rest of this section have been tailored in a similar way.
The security association (SA) between two end points is established by exchanging IKE messages. The Parse Payload function parses the messages in a human-readable format. The logging can be turned on by editing the /etc/isakmpd.conf file. The logging entry in the /etc/isakmpd.conf file looks similar to the following:
information
The type of IKE payloads that Parse Payload logs depends on the content of the IKE message. Examples include SA Payload, Key Exchange Payload, Certificate Request Payload, Certificate Payload, and Signature Payload. The following is an example of a Parse Payload log in which an ISAKMP_MSG_HEADER is followed by five payloads:
ISAKMP_MSG_HEADER
Icookie : 0x9e539a6fd4540990, Rcookie : 0x0000000000000000
Next Payload : 1(SA), Maj Ver : 1, Min Ver : 0
Xchg Type : 4 (Aggressive), Flag= 0, Encr : No,COMMIT : No
Msg ID : 0x00000000
len : 0x10e(270)
SA Payload:
Next Payload : 4(Key Exchange), Payload len : 0x34(52)
DOI : 0x1(INTERNET)
bitmask : 1(SIT_IDENTITY_ONLY
Proposal Payload:
Next Payload : 0(NONE), Payload len : 0x28(40)
Proposal # : 0x1(1), Protocol-ID : 1(ISAKMP)
SPI size : 0x0(0), # of Trans : 0x1(1)
Transform Payload:
Next Payload : 0(NONE), Payload len : 0x20(32)
Trans # : 0x1(1), Trans.ID : 1(KEY_IKE)
Attr : 1(Encr.Alg ), len=0x2(2)
Value=0x1(1),(DES-cbc)
Attr : 2(Hash Alg ), len=0x2(2)
Value=0x1(1),(MD5)
Attr : 3(Auth Method ), len=0x2(2)
Value=0x3(3),(RSA Signature)
Attr : 4(Group Desc ), len=0x2(2)
Value=0x1(1),(default 768-bit MODP group)
Attr : 11(Life Type ), len=0x2(2)
Value=0x1(1),(seconds)
Attr : 12(Life Duration), len=0x2(2)
Value=0x7080(28800)
Key Payload:
Next Payload : 10(Nonce), Payload len : 0x64(100)
Key Data :
33 17 68 10 91 1f ea da 38 a0 22 2d 84 a3 5d 5d
a0 e1 1f 42 c2 10 aa 8d 9d 14 0f 58 3e c4 ec a3
9f 13 62 aa 27 d8 e5 52 8d 5c c3 cf d5 45 1a 79
8a 59 97 1f 3b 1c 08 3e 2a 55 9b 3c 50 cc 82 2c
d9 8b 39 d1 cb 39 c2 a4 05 8d 2d a1 98 74 7d 95
ab d3 5a 39 7d 67 5b a6 2e 37 d3 07 e6 98 1a 6b
Nonce Payload:
Next Payload : 5(ID), Payload len : 0xc(12)
Nonce Data:
6d 21 73 1d dc 60 49 93
ID Payload:
Next Payload : 7(Cert.Req), Payload len : 0x49(73)
ID type : 9(DER_DN), Protocol : 0, Port = 0x0(0)
Certificate Request Payload:
Next Payload : 0(NONE), Payload len : 0x5(5)
Certificate Encoding Type: 4(X.509 Certificate - Signature)
Within each payload, there is a Next Payload field that points to the payload following the current payload. If the current payload is the last one in the IKE message, the Next Payload field has the value of zero (None).
Each Payload in the example has information pertaining to the negotiations that are going on. For example, the SA payload has the Proposal and Transform Payloads, which in turn show the encryption algorithm, authentication mode, hash algorithm, SA life type, and SA duration that the initiator is proposing to the responder.
Also, the SA Payload consists of one or more Proposal Payloads and one or more Transform Payloads. The Next Payload field for Proposal Payload has a value of either zero if it is the only Proposal Payload or a value of two if it is followed by one more Proposal Payloads. Similarly the Next Payload field for a Transform Payload has a value of zero if it is the only Transform Payload or a value of 3 if it is followed by one more Transform Payloads, as shown in the following example:
ISAKMP_MSG_HEADER
Icookie : 0xa764fab442b463c6, Rcookie : 0x0000000000000000
Next Payload : 1(SA), Maj Ver : 1, Min Ver : 0
Xchg Type : 2 (ID protected), Flag= 0, Encr : No,COMMIT : No
Msg ID : 0x00000000
len : 0x70(112)
SA Payload:
Next Payload : 0(NONE), Payload len : 0x54(84)
DOI : 0x1(INTERNET)
bitmask : 1(SIT_IDENTITY_ONLY
Proposal Payload:
Next Payload : 0(NONE), Payload len : 0x48(72)
Proposal # : 0x1(1), Protocol-ID : 1(ISAKMP)
SPI size : 0x0(0), # of Trans : 0x2(2)
Transform Payload:
Next Payload : 3(Transform), Payload len : 0x20(32)
Trans # : 0x1(1), Trans.ID : 1(KEY_IKE)
Attr : 1(Encr.Alg ), len=0x2(2)
Value=0x5(5),(3DES-cbc)
Attr : 2(Hash Alg ), len=0x2(2)
Value=0x1(1),(MD5)
Attr : 3(Auth Method ), len=0x2(2)
Value=0x1(1),(Pre-shared Key)
Attr : 4(Group Desc ), len=0x2(2)
Value=0x1(1),(default 768-bit MODP group)
Attr : 11(Life Type ), len=0x2(2)
Value=0x1(1),(seconds)
Attr : 12(Life Duration), len=0x2(2)
Value=0x7080(28800)
Transform Payload:
Next Payload : 0(NONE), Payload len : 0x20(32)
Trans # : 0x2(2), Trans.ID : 1(KEY_IKE)
Attr : 1(Encr.Alg ), len=0x2(2)
Value=0x1(1),(DES-cbc)
Attr : 2(Hash Alg ), len=0x2(2)
Value=0x1(1),(MD5)
Attr : 3(Auth Method ), len=0x2(2)
Value=0x1(1),(Pre-shared Key)
Attr : 4(Group Desc ), len=0x2(2)
Value=0x1(1),(default 768-bit MODP group)
Attr : 11(Life Type ), len=0x2(2)
Value=0x1(1),(seconds)
Attr : 12(Life Duration), len=0x2(2)
Value=0x7080(28800)
The IKE Message Header of a Parse Payload log shows the exchange type (Main Mode or Aggressive Mode), the length of the entire message, the message identifier, etc.
The Certificate Request Payload requests a certificate from the responder. The responder sends the certificate in a separate message. The following example shows the Certificate Payload and Signature Payload that are sent to a peer as a part of an SA negotiation. The certificate data and the signature data are printed in hex format.
ISAKMP_MSG_HEADER
Icookie : 0x9e539a6fd4540990, Rcookie : 0xc7e0a8d937a8f13e
Next Payload : 6(Certificate), Maj Ver : 1, Min Ver : 0
Xchg Type : 4 (Aggressive), Flag= 0, Encr : No,COMMIT : No
Msg ID : 0x00000000
len : 0x2cd(717)
Certificate Payload:
Next Payload : 9(Signature), Payload len : 0x22d(557)
Certificate Encoding Type: 4(X.509 Certificate - Signature)
Certificate: (len 0x227(551) in bytes
82 02 24 30 82 01 8d a0 03 02 01 02 02 05 05 8e
fb 3e ce 30 0d 06 09 2a 86 48 86 f7 0d 01 01 04
05 00 30 5c 31 0b 30 09 06 03 55 04 06 13 02 46
49 31 24 30 22 06 03 55 04 0a 13 1b 53 53 48 20
43 6f 6d 6d 75 6e 69 63 61 74 69 6f 6e 73 20 53
65 63 75 72 69 74 79 31 11 30 0f 06 03 55 04 0b
13 08 57 65 62 20 74 65 73 74 31 14 30 12 06 03
55 04 03 13 0b 54 65 73 74 20 52 53 41 20 43 41
30 1e 17 0d 39 39 30 39 32 31 30 30 30 30 30 30
5a 17 0d 39 39 31 30 32 31 32 33 35 39 35 39 5a
30 3f 31 0b 30 09 06 03 55 04 06 13 02 55 53 31
10 30 0e 06 03 55 04 0a 13 07 49 42 4d 2f 41 49
58 31 1e 30 1c 06 03 55 04 03 13 15 62 61 72 6e
65 79 2e 61 75 73 74 69 6e 2e 69 62 6d 2e 63 6f
6d 30 81 9f 30 0d 06 09 2a 86 48 86 f7 0d 01 01
01 05 00 03 81 8d 00 30 81 89 02 81 81 00 b2 ef
48 16 86 04 7e ed ba 4c 14 d7 83 cb 18 40 0a 3f
55 e9 ad 8f 0f be c5 b6 6d 19 ec de 9b f5 01 a6
b9 dd 64 52 34 ad 3d cd 0d 8e 82 6a 85 a3 a8 1c
37 e4 00 59 ce aa 62 24 b5 a2 ea 8d 82 a3 0c 6f
b4 07 ad 8a 02 3b 19 92 51 88 fb 2c 44 29 da 72
41 ef 35 72 79 d3 e9 67 02 b2 71 fa 1b 78 13 be
f3 05 6d 10 4a c7 d5 fc fe f4 c0 b8 b8 fb 23 70
a6 4e 16 5f d4 b1 9e 21 18 82 64 6d 17 3b 02 03
01 00 01 a3 0f 30 0d 30 0b 06 03 55 1d 0f 04 04
03 02 07 80 30 0d 06 09 2a 86 48 86 f7 0d 01 01
04 05 00 03 81 81 00 75 a4 ee 9c 3a 18 f2 de 5d
67 d4 1c e4 04 b4 e5 b8 5e 9f 56 e4 ea f0 76 4a
d0 e4 ee 20 42 3f 20 19 d4 25 57 25 70 0a ea 41
81 3b 0b 50 79 b5 fd 1e b6 0f bc 2f 3f 73 7d dd
90 d4 08 17 85 d6 da e7 c5 a4 d6 9a 2e 8a e8 51
7e 59 68 21 55 4c 96 4d 5a 70 7a 50 c1 68 b0 cf
5f 1f 85 d0 12 a4 c2 d3 97 bf a5 42 59 37 be fe
9e 75 23 84 19 14 28 ae c4 c0 63 22 89 47 b1 b6
f4 c7 5d 79 9d ca d0
Signature Payload:
Next Payload : 0(NONE), Payload len : 0x84(132)
Signature: len 0x80(128) in bytes
9d 1b 0d 90 be aa dc 43 95 ba 65 09 b9 00 6d 67
b4 ca a2 85 0f 15 9e 3e 8d 5f e1 f0 43 98 69 d8
5c b6 9c e2 a5 64 f4 ef 0b 31 c3 cb 48 7c d8 30
e3 a2 87 f4 7c 9d 20 49 b2 39 00 fa 8e bf d9 b0
7d b4 8c 4e 19 3a b8 70 90 88 2c cf 89 69 5d 07
f0 5a 81 58 2e 15 40 37 b7 c8 d6 8c 5c e2 50 c3
4d 19 7e e0 e7 c7 c2 93 42 89 46 6b 5f f8 8b 7d
5b cb 07 ea 36 e5 82 9d 70 79 9a fe bd 6c 86 36
| Error: | The cpsd (Certificate Proxy Server daemon) does not
start. An entry similar to the following appears in the log file:
Sep 21 16:02:00 ripple CPS[19950]: Init():LoadCaCerts() failed, rc=-12 Problem: The certificate database has not opened or has not been found. To Fix: Ensure that the IBM Key Manager certificate databases are present in /etc/security. The following files make up the database: ikekey.crl, ikekey.kdb, ikekey.rdb, ikekey.sth. If only the ikekey.sth file is missing, the stash password option was not selected when the IBM Key Manager database was created. The password must be stashed to enable using digital certificates with IP Security. (See Creating a Key Database for more information.) |
| Error: | IBM Key Manager gives the following error when receiving a
certificate:
Invalid Base64-encoded data was found Problem: Superfluous data has been found in the certificate file or else data was lost or corrupted. To Fix: The 'DER' Encoded Certificate should be contained within the following strings (shown below). No other characters should precede or follow other than the BEGIN and END CERTIFICATE strings.
-----BEGIN CERTIFICATE----- MIICMTCCAZqgAwIBAgIFFKZtANowDQYJKoZIhvcNAQEFBQAwXDELMAkGA1UEBhMC RkkxJDAiBgNVBAoTG1NTSCBDb21tdW5pY2F0aW9ucyBTZWN1cml0eTERMA8GA1UE CxMIV2ViIHRlc3QxFDASBgNVBAMTC1Rlc3QgUlNBIENBMB4XDTk5MDkyMTAwMDAw MFoXDTk5MTAyMTIzNTk1OVowOzELMAkGA1UEBhMCVVMxDDAKBgNVBAoTA0lCTTEe MBwGA1UEAxMVcmlwcGxlLmF1c3Rpbi5pYm0uY29tMIGfMA0GCSqGSIb3DQEBAQUA A4GNADCBiQKBgQC5EZqo6n7tZrpAL6X4L7mf4yXQSm+m/NsJLhp6afbFpPvXgYWC wq4pvOtvxgum+FHrE0gysNjbKkE4Y6ixC9PGGAKHnhM3vrmvFjnl1G6KtyEz58Lz BWW39QS6NJ1LqqP1nT+y3+Xzvfv8Eonqzno8mglCWMX09SguLmWoU1PcZQIDAQAB oyAwHjALBgNVHQ8EBAMCBaAwDwYDVR0RBAgwBocECQNhhzANBgkqhkiG9w0BAQUF AOBgQA6bgp4Zay34/fyAlyCkNNAYJRrN3Vc4NHN7IGjUziN6jK5UyB5zL37FERW hT9ArPLzK7yEZs+MDNvB0bosyGWEDYPZr7EZHhYcoBP4/cd0V5rBFmA8Y2gUthPi Ioxpi4+KZGHYyLqTrm+8Is/DVJaQmCGRPynHK35xjT6WuQtiYg== -----END CERTIFICATE----- The following options can help you diagnose and solve this problem.
|
| Error: | IBM Key Manager gives the following error when receiving a personal
certificate:
No request key was found for the certificate Problem: A Personal Certificate Request does not exist for the personal certificate being received. To Fix: Create the Personal Certificate Request again and request a new certificate. |
| Error: | Web-based System Manager gives the following error when you configure an
IKE tunnel:
Error 171 in the Key Management (Phase 1) Tunnel operation: PUT_IRL_FAILED Problem: One cause for this error is the host identity type, which is configured on the IKE dialog (Identification tab), is invalid. This can happen when the host identity type selected from the pull-down list does not logically match the type entered in the Host Identity field. For example, if you select a host identity type of X500 Distinguished Name, you need to enter a properly formatted distinguished name in the Host Identity field. To Fix: Ensure the distinguished name you enter is correct for the type selected in the host identity pull-down list. |
| Error: | An IKE negotiation fails and an entry similar to the following appears in
the log file:
inet_cert_service::channelOpen():clientInitIPC():error,rc =2 (No such file or directory) Problem: The cpsd is not running or has died To Fix: Start IP Security using Web-based System Manager. This action also starts the appropriate daemons. |
| Error: | An IKE negotiation fails and an entry similar to the following appears in
the log file:
CertRepo::GetCertObj: DN Does Not Match: ("/C=US/O=IBM/CN=ripple.austin.ibm.com")
Problem: The X.500 Distinguished Name (DN) entered while defining the IKE tunnel does not match the X.500 DN in the personal certificate. To Fix: Change the IKE tunnel definition in Web-based System Manager to match the distinguished name in the certificate. |
| Error: | While defining IKE tunnels in Web-based System Manager, the Digital
certificate check box is disabled under the Authentication
Method tab.
Problem: The policy associated with this tunnel does not use RSA signature mode authentication. To Fix: Change the transform of the associated policy to use the RSA signature authentication method. For example, you can choose IBM_low_CertSig as a key management policy when defining a IKE tunnel. |
Tracing is a debug facility for tracing kernel events. Traces can be used to get more specific information about events or errors occurring in the kernel filter and tunnel code.
SMIT has an IP Security trace facility available through the Advanced IP Security Configuration menu. The information captured by this trace facility includes information on Error, Filter, Filter Information, Tunnel, Tunnel Information, Capsulation/Decapsulation, Capsulation Information, Crypto, and Crypto Information. By design, the error trace hook provides the most critical information. The info trace hook can generate a lot of information and may have an impact on system performance. This tracing will provide clues to you as to what a problem may be. Tracing information will also be required when speaking with an IBM IP Security Technician. To access the tracing facility, use the SMIT fast path smit ips4_tracing (for IP Version 4) or smit ips6_tracing (for IP Version 6).
You can issue the ipsecstat command to generate the following sample report. This sample report shows that the IP Security devices are in the available state, that there are three authentication algorithms installed, three encryption algorithms installed, and that there is a current report of packet activity. This information may be useful to you in determining where a problem exists if you are troubleshooting your IP Security traffic.
IP Security Devices: ipsec_v4 Available ipsec_v6 Available Authentication Algorithm: HMAC_MD5 -- Hashed MAC MD5 Authentication Module HMAC_SHA -- Hashed MAC SHA Hash Authentication Module KEYED_MD5 -- Keyed MD5 Hash Authentication Module Encryption Algorithm: CDMF -- CDMF Encryption Module DES_CBC_4 -- DES CBC 4 Encryption Module DES_CBC_8 -- DES CBC 8 Encryption Module 3DES_CBC -- Triple DES CBC Encryption Module IP Security Statistics - Total incoming packets: 1106 Incoming AH packets:326 Incoming ESP packets: 326 Srcrte packets allowed: 0 Total outgoing packets:844 Outgoing AH packets:527 Outgoing ESP packets: 527 Total incoming packets dropped: 12 Filter denies on input: 12 AH did not compute: 0 ESP did not compute:0 AH replay violation:0 ESP replay violation: 0 Total outgoing packets dropped:0 Filter denies on input:0 Tunnel cache entries added: 7 Tunnel cache entries expired: 0 Tunnel cache entries deleted: 6
Note: Beginning with AIX 4.3.3, CDMF support has been removed because DES is now available world wide. Reconfigure any tunnels that use CDMF to use DES or Triple DES.