This section discusses the following topics:
The operating system uses IP Security (IPsec), which is an open, standard security technology developed by the Internet Engineering Task Force (IETF). IPsec provides cryptography-based protection of all data at the IP layer of the communications stack. No changes are needed for existing applications. IPsec is the industry-standard network-security framework chosen by the IETF for both the IP Version 4 and 6 environments.
IPsec protects your data traffic using the following cryptographic techniques:
Authentication algorithms prove the identity of the sender and data integrity by using a cryptographic hash function to process a packet of data (with the fixed IP header fields included) using a secret key to produce a unique digest. On the receiver side, the data is processed using the same function and key. If either the data has been altered or the sender key is not valid, the datagram is discarded.
Encryption uses a cryptographic algorithm to modify and randomize the data using a certain algorithm and key to produce encrypted data known as cyphertext. Encryption makes the data unreadable while in transit. After it is received, the data is recovered using the same algorithm and key (with symmetric encryption algorithms). Encryption must occur with authentication to verify the data integrity of the encrypted data.
These basic services are implemented in IPsec by the use of the Encapsulating Security Payload (ESP) and the Authentication Header (AH). ESP provides confidentiality by encrypting the original IP packet, building an ESP header, and putting the cyphertext in the ESP payload.
The AH can be used alone for authentication and integrity-checking if confidentiality is not an issue. With AH, the static fields of the IP header and the data have a hash algorithm applied to compute a keyed digest. The receiver uses its key to compute and compare the digest to make sure the packet is unaltered and the sender's identity is authenticated.
The IP Security feature of this operating system provides the following functions:
The following features are available with Internet Key Exchange (beginning with AIX 4.3.2):
The building block on which secure communications is built is a concept known as a security association. Security associations relate a specific set of security parameters to a type of traffic. With data protected by IP Security, a separate security association exists for each direction and for each header type, AH or ESP. The information contained in the security association includes the IP addresses of the communicating parties, a unique identifier known as the Security Parameters Index (SPI), the algorithms selected for authentication or encryption, the authentication and encryption keys, and the key lifetimes. The following figure shows the security associations between Host A and Host B.
The goal of key management is to negotiate and compute the security associations that protect IP traffic.
To set up a secure communication between two hosts, security associations must be negotiated and managed during the use of the tunnel. The following types of tunnels are supported, each using a different key management technique:
IKE Tunnels are based on the ISAKMP/Oakley (Internet Security Association and Key Management Protocol) standards developed by the IETF. With this protocol, security parameters are negotiated and refreshed, and keys are exchanged securely. The following types of authentication are supported: preshared key and X.509v3 digital certificate signatures.
The negotiation uses a two-phase approach. The first phase authenticates the communicating parties, and specifies the algorithms to be used for securely communicating in phase 2. During phase 2, IP Security parameters to be used during data transfer are negotiated, security associations and keys are created and exchanged.
The following table shows the authentication algorithms that can be used with the AH and ESP security protocols for IKE tunnel support.
| Algorithm | AH IP Version 4 & 6 | ESP IP Version 4 & 6 |
|---|---|---|
| HMAC MD5 | X | X |
| HMAC SHA1 | X | X |
| DES CBC 8 | X | |
| Triple DES CBC | X | |
| ESP Null | X |
Manual tunnels provide backward compatibility, and they interoperate with machines that do not support IKE key management protocols. The disadvantage of manual tunnels is that the key values are static. The encryption and authentication keys are the same for the life of the tunnel and must be manually updated.
The following table shows the authentication algorithms that can be used with the AH and ESP security protocols for manual tunnel support.
| Algorithm | AH IP Version 4 | AH IP Version 6 | ESP IP Version 4 | ESP IP Version 6 |
|---|---|---|---|---|
| HMAC MD5 | X | X | X | X |
| HMAC SHA1 | X | X | X | X |
| Triple DES CBC | X | X | ||
| DES CBC 8 | X | X | ||
| DES CBC 4 | X | X |
Because IKE tunnels offer more effective security, IKE is the preferred key management method.
Filtering is a basic function in which incoming and outgoing packets can be accepted or denied based on a variety of characteristics. This allows a user or system administrator to configure the host to control the traffic between this host and other hosts. Filtering is done on a variety of packet properties, such as source and destination addresses, IP version (4 or 6), subnet masks, protocol, port, routing characteristics, fragmentation, interface, and tunnel definition.
Rules, known as filter rules, are used to associate certain kinds of traffic with a particular tunnel. In a basic configuration for manual tunnels, when a user defines a host-to-host tunnel, filter rules are autogenerated to direct all traffic from that host through the secure tunnel. If more specific types of traffic are desired (for instance, subnet to subnet), the filter rules can be edited or replaced to allow precise control of the traffic using a particular tunnel.
For IKE tunnels, the filter rules are also automatically generated and inserted in the filter table once the tunnel is activated.
Similarly, when the tunnel is modified or deleted, the filter rules for that tunnel are automatically deleted, which simplifies IP Security configuration and helps reduce human error. Tunnel definitions can be propagated and shared among machines and firewalls using import and export utilities, which is helpful in the administration of a large number of machines.
Filter rules associate particular types of traffic with a tunnel, but data being filtered does not necessarily need to travel in a tunnel. This aspect of filter rules lets the operating system provide basic firewall functionality to those who want to restrict traffic to or from their machine in an intranet or in a network that does not have the protection of a true firewall. In this scenario, filter rules provide a second barrier of protection around a group of machines.
After the filter rules are generated, they are stored in a table and loaded into the kernel. When packets are ready to be sent or received from the network, the filter rules are checked in the list from top to bottom to determine whether the packet should be permitted, denied, or sent through a tunnel. The criteria of the rule is compared to the packet characteristics until a match is found or the default rule is reached.
The IP Security function also implements filtering of non-secure packets based on very granular, user-defined criteria, which allows the control of IP traffic between networks and machines that do not require the authentication or encryption properties of IP Security.
IP Security supports the use of X.509 Version 3 digital certificates. The Key Manager tool manages certificate requests, maintains the key database, and performs other administrative functions.
Digital certificates are described in Digital Certificate Configuration. The Key Manager and its functions are described in Using the IBM Key Manager Tool
A virtual private network (VPN) securely extends a private intranet across a public network such as the Internet. VPNs convey information across what is essentially a private tunnel through the Internet to and from remote users, branch offices, and business partners/suppliers. Companies can opt for Internet access through Internet service providers (ISPs) using direct lines or local telephone numbers and eliminate more expensive leased lines, long-distance calls, and toll-free telephone numbers. A VPN solution can use the IPsec security standard because IPsec is the IETF-chosen industry standard network security framework for both the IP Version 4 and 6 environments, and no changes are needed for existing applications.
A recommended resource for planning the implementation of a VPN in AIX is Chapter 9 of A Comprehensive Guide to Virtual Private Networks, Volume III: Cross-Platform Key and Policy Management, ISBN SG24-5309-00. This guide is also available on the Internet World Wide Web at http://www.redbooks.ibm.com/redbooks/SG245309.html.