System Management Guide: Communications and Networks

TCP/IP Local Area Network Adapter Cards

The topics discussed in this section are:

The network adapter card is the hardware that is physically attached to the network cabling. It is responsible for receiving and transmitting data at the physical level. The network adapter card is controlled by the network adapter device driver.

A machine must have one network adapter card (or connection) for each network (not network type) to which it connects. For instance, if a host attaches to two token-ring networks, it must have two network adapter cards.

TCP/IP uses the following network adapter cards and connections:

Standard Ethernet Version 2
IEEE 802.3
Token-ring
Asynchronous adapters and native serial ports (described in AIX 5L Version 5.1 Asynchronous Communications Guide)
Fiber Distributed Data Interface (FDDI)
Serial Optical Channel Converter (described in AIX 5L Version 5.1 Kernel Extensions and Device Support Programming Concepts)
Asynchronous Transfer Mode (ATM).

The Ethernet and 802.3 network technologies use the same type of adapter.

Each machine provides a limited number of expansion slots, some or all of which you might wish to use for communications adapters. Additionally, each machine supports a limited number of communications adapters of a given type. Each machine supports up to eight Ethernet/802.3 adapters, up to eight token-ring adapters, and one asynchronous adapter card with up to 64 connections. Within these limits (software limitations), you can install any combination of these adapters up to the total number of expansion slots available in your machine (hardware limitations).

Only one Transmission Control Protocol/Internet Protocol (TCP/IP) interface is configurable regardless of the number of Serial Optical Channel Converters supported by the system. The Serial Optical device driver makes use of both channel converters even though only one logical TCP/IP interface is configured.

Installing a Network Adapter

To install a network adapter:

Shut down the computer. See the shutdown command for information on how to shut down a system.
Turn off the computer power.
Remove the computer cover.
Find a free slot on the Micro Channel bus and insert the network adapter. Be careful to seat the adapter properly in the slot.
Replace the computer cover.
Restart the computer.

Configuring and Managing Adapters

To configure and manage token-ring or Ethernet adapters, use the tasks in the following table.

Configuring and Managing Adapters Tasks
Task	SMIT Fast Path	Command or File	Web-based System Manager Management Environment⁵
Configure an Adapter	smit chgtok (token ring) smit chgenet (Ethernet)	Determine adapter name:¹ `lsdev -C -c adapter -t tokenring -H` or `lsdev -C -c adapter -t ethernet -H` Reset ring speed (token ring) or connector type (Ethernet), if necessary. For example: `chdev -l tok0 -a ring_speed=16 -P` or `chdev -l ent0 -a bnc_select=dix -P`
Determining a Network Adapter Hardware Address	smit chgtok (token ring) smit chgenet (Ethernet)	`lscfg -l tok0 -v` (token ring)² `lscfg -l ent0 -v` (Ethernet)²
Setting an Alternate Hardware Address	smit chgtok (token ring) smit chgenet (Ethernet)	Define the alternate hardware address. For example, for token ring:^2,3 `chdev -l tok0 -a alt_addr=0X10005A4F1B7F` For Ethernet:^2,3 `chdev -l ent0 -a alt_addr=0X10005A4F1B7F -p` Begin using alternate address, for token ring:⁴ `chdev -l tok0 -a use_alt_addr=yes` For Ethernet:⁴ `chdev -l ent0 -a use_alt_addr=yes`

Notes:

The name of a network adapter can change if you move it from one slot to another or remove it from the system. If you ever move the adapter, issue the diag -a command to update the configuration database.
Substitute your adapter name for tok0 and ent0.
Substitute your hardware address for 0X10005A4F1B7F.
After performing this procedure, you might experience a disruption of communication with other hosts until they flush their Address Resolution Protocol (ARP) cache and obtain the new hardware address of this host.
These tasks are not available in Web-based System Manager Management Environment.

Configuring and Using Virtual Local Area Networks (VLANs)

VLANs (Virtual Local Area Networks) can be thought of as logical broadcast domains. A VLAN splits up groups of network users on a real physical network onto segments of logical networks. This implementation supports the IEEE 802.1Q VLAN tagging standard with the capability to support multiple VLAN IDs running on Ethernet adapters. Each VLAN ID is associated with a separate Ethernet interface to the upper layers (IP, etc.) and creates unique logical Ethernet adapter instances per VLAN, for example ent1, ent2 and so on.

The IEEE 802.1Q VLAN support can be configured over any supported Ethernet adapters. The adapters must be connected to a switch that supports IEEE 802.1Q VLAN.

You can configure multiple VLAN logical devices on a single system. Each VLAN logical devices constitutes an additional Ethernet adapter instance. These logical devices can be used to configure the same Ethernet IP interfaces as are used with physical Ethernet adapters. As such, the no option, ifsize (default 8), needs to be increased to include not only the Ethernet interfaces for each adapter, but also any VLAN logical devices that are configured. See the no command documentation.

Each VLAN can have a different maximum transmission unit (MTU) value even if sharing a single physical Ethernet adapter.

VLAN support is managed through SMIT. Type the smit vlan fast path from the command line and make your selection from the main VLAN menu. Online help is available.

After you configure VLAN, configure the IP interface, for example, en1 for standard Ethernet or et1 for IEEE 802.3, using Web-based System Manager, SMIT, or commands.

Notes:
If you try to configure a VLAN ID value that is already in use for the specified adapter, the configuration fails with the following error:
Method error (/usr/lib/methods/chgvlan):
  0514-018 The values specified for the following attributes
           are not valid:
           vlan_tag_id    VLAN Tag ID
If a user (for example, IP interface) is currently using the VLAN logical device, any attempt to remove the VLAN logical device fails. A message similar to the following displays:
Method error (/usr/lib/methods/ucfgcommo):
        0514-062 Cannot perform the requested function because the
                 specified device is busy.
To remove the logical VLAN device, first detach the user. For example, if the user is IP interface en1, then you can use the following command:
ifconfig en1 detach
Then remove the network interface using the SMIT TCP/IP menus.
If a user (for example, IP interface) is currently using the VLAN logical device, any attempt to change the VLAN characteristic (VLAN tag ID or base adapter) fails. A message similar to the following displays:
Method error (/usr/lib/methods/chgvlan):
        0514-062 Cannot perform the requested function because the
                 specified device is busy.
To change the logical VLAN device, first detach the user. For example, if the user is the IP interface en1, you could use the following command:
ifconfig en1 detach
Then change the VLAN and add the network interface again using the SMIT TCP/IP menus.

Troubleshooting

tcpdump and trace can be used to troubleshoot the VLAN. The trace hook ID for each type of transmit packet follows:

transmit packets	3FD
receive packets	3FE
other events	3FF

The entstat command gives the aggregate statistics of the physical adapter for which the VLAN is configured. It does not provide the individual statistics for that particular VLAN logical device.

Restrictions

Remote dump is not supported over a VLAN. Also, VLAN logical devices cannot be used to create a Cisco Systems' Etherchannel.

Using ATM Adapters

Asynchronous Transfer Mode (ATM) is an international standard that defines a high-speed networking method to transport any mixture of voice, video, and traditional computer data across local, municipal, and wide-area networks (LANs, MANs, and WANs). ATM adapters provide full-duplex connectivity for RS/6000 servers or clients using permanent virtual circuits (PVCs) and switched virtual circuits (SVCs). The PVC and SVC implementations are designed to be compliant with the ATM Forum specifications. The maximum number of virtual circuits supported depends on the adapter. Most adapters support at least 1024 virtual circuits.

ATM Technology

Asynchronous Transfer Mode (ATM) is a cell-switching, connection-oriented technology. In ATM networks, end stations attach to the network using dedicated full duplex connections. The ATM networks are constructed using switches, and switches are interconnected using dedicated physical connections. Before any data transfers can begin, end-to-end connections must be established. Multiple connections can and do exist on a single physical interface. Sending stations transmit data by segmenting Protocol Data Units (PDUs) into 53-byte cells. Payload stays in the form of cells during network transport. Receiving stations reassemble cells into PDUs. The connections are identified using a virtual path identifier (VPI) and a virtual channel identifier (VCI). The VPI field occupies one byte in the ATM cell five-byte header; whereas, the VCI field occupies two bytes in the ATM cell five-byte header. Basically, a VPI:VCI pair identifies the source of the ATM cell. The function of the ATM switch is to recognize the source of the cell, determine the next hop, and output the cell to a port. The VPI:VCI changes on a hop-by-hop basis. Thus, VPI:VCI values are not universal. Each virtual circuit is described as a concatenation of VPI:VCI values across the network.

ATM Connections

ATM architecture has two kinds of virtual circuits: permanent (PVCs) and switched (SVCs).

Permanent Virtual Circuits

PVCs are statically and manually configured. The switches forming the ATM network must first be set up to recognize the VPI:VCI combination of each endpoint and to route the endpoint ATM cells to the destination endpoint through the ATM network. Once a link connection through the network has been established from one endpoint to another, ATM cells can be transmitted through the ATM network and ATM switches. The network switches translate the VPI:VCI values in the appropriate way so as to route the cell to its destination.

Switched Virtual Circuits

SVCs are dynamically set up on an as needed basis. The ATM end stations are assigned 20-byte addresses. SVCs use a control plane and a data plane.
The control plane uses a signaling channel VPI:VCI 0:5.
SVCs involve on demand call setup, whereby an ATM station sends information elements specifying the destination ATM address (and optionally, the source ATM address). In general, calling station, network, and called station participate in a negotiation. Finally, a call is either accepted or rejected. If a call is accepted, network assigns VPI:VCI values for the data plane to the calling station and called station. In the control plane, the ATM network routes (or switches) signaling packets on the basis of the ATM addresses. While these packets are being routed, the switches set up data plane cell routing tables. In the data plane, ATM networks switch cells on the basis of VPI:VCI much like in the case of PVCs. When data transfer is over, connection is terminated.

The ATM address is constructed by registering with the ATM network and by acquiring the most significant 13 bytes. The next six bytes contain the adapter's factory-assigned, unique MAC address. The least significant byte is the selector. Use of this byte is left to the discretion of the end station. ATM networks do not interpret this byte.

TCP/IP over ATM

The Internet Engineering Task Force RFC1577: Classical IP and ARP over ATM standard specifies the mechanism for implementing Internet Protocol (IP) over ATM. Since ATM is connection-oriented technology and IP is a datagram-oriented technology, mapping the IP over ATM is not trivial.

In general, the ATM network is divided into logical IP subnetworks (LISs). Each LIS is comprised of some number of ATM stations. LISs are analogous to traditional LAN segments. LISs are interconnected using routers. A particular adapter (on an ATM station) can be part of multiple LISs. This feature can be very useful for implementing routers.

RFC1577 specifies RFC1483, which specifies logical link control/Sub-Network Access Protocol (LLC/SNAP) encapsulation as the default. In PVC networks for each IP station, all PVCs must be manually defined by configuring VPI:VCI values. If LLC/SNAP encapsulation is not being used, the destination IP address associated with each VPI:VCI must be defined. If LLC/SNAP encapsulation is being used, the IP station can learn the remote IP address by an InARP mechanism.

For SVC networks, RFC1577 specifies an ARP server per LIS. The purpose of the ARP server is to resolve IP addresses into ATM addresses without using broadcasts. Each IP station is configured with the ATM address of the ARP server. IP stations set up SVCs with the ARP server, which in turn, sends InARP requests to the IP stations. Based on InARP reply, an ARP server sets up IP to ATM address maps. IP stations send ARP packets to the ARP server to resolve addresses, which returns ATM addresses. IP stations then set up a SVC to the destination station and data transfer begins. The ARP entries in IP stations and the ARP server age based on a well defined mechanism. For both the PVC and SVC environments, each IP station has at least one virtual circuit per destination address.

The Internet Engineering Task Force RFC2225 adds the support of ATM ARP Request Address list to RFC1577. The ATM ARP Request Address list is a list containing one or more ATM addresses of individual ATM ARP servers located within the LIS. The RFC2225 client eliminates the single point of failure associated with the 1577 clients' ATM ARP services. The 2225 clients have the ability to switch to backup ARP servers when the current ATM ARP server fails.

RS/6000 sets the first entry in the ATM ARP Request Address list as the Primary ATM ARP server and the rest of the entries as Secondary ATM ARP servers.

The client will always try to use the Primary ATM ARP server. If the effort to connect to the Primary ATM ARP server fails, the client tries to connect to the first Secondary server (the position in the ATM ARP Request Address list determines the order of the Secondary ATM ARP server). If the connection to the first Secondary ATM ARP server fails, the client tries to contact the next Secondary ATM ARP server in the list. This process continues until the connection is successful.

If the connection to the Primary ATM ARP server fails, regardless of which Secondary ATM ARP server it is connected to or attempting to connect to, the client continues to retry the Primary ATM ARP server every 15 minutes. If it finally connects to the Primary ATM ARP server, then the connection to the current Secondary ATM ARP server is dropped.

The ATM ARP Request Address list is entered manually either through SMIT or by using the ifconfig command. The ATM ARP Request Address list cannot be configured with the Management Information Base (MIB).

PVC Network

Use the "Representative ATM Network" figure as an example to configure your network.

Within the "Representative ATM Network" figure, one logical IP subnet is represented by dashed lines from each host to the switch. The other IP subnet is represented by solid lines from each host to the switch.

Figure 3-12. Representative ATM Network. This illustration depicts an ATM network laid out in a typical star topography. In the center of the star is the ATM switch. Numbered IP hosts are branched off of the switch as are links to other ATM switches and one ATM gateway box and adapter.

The following Representative Host Configuration table indicates how hosts H3 and H4 are configured to communicate with a gateway and with each host on its own logical IP subnet.

Representative Host Configuration
Network Interface Driver	VPI:VCI	Comment
Host H3
at0	0:40	Connection to 128.10.1.5 (gateway)
at0	0:42	Connection to 128.10.1.2
at0	0:43	Connection to 128.10.1.3
Host H4
at0	0:50	Connection to 128.10.2.5 (gateway)
at0	0:52	Connection to 128.10.2.2
at0	0:53	Connection to 128.10.2.3
at0	0:54	Connection to 128.10.2.4

To reach hosts on another logical IP subnet, only a VPI:VCI connection to the gateway needs to be created. (The VPI:VCIs are for illustration purposes only.)

The ATM gateway box has one ATM with two IP addresses sharing the same physical cable.

SVC Network

Using the "Representative ATM Network" figure as an example, imagine that host H3 wants to call H4. H1 is the ARP server for subnet 1 and H6 is the ARP server for subnet 2. Assuming a subnet mask of 255.255.255.0, stations with addresses of 128.10.1.X are members of one subnet, whereas stations with addresses of 128.10.2.X are members of a second subnet. See the following list of representative host configurations using SVCs.

Figure 3-13. Representative ATM Network. This illustration depicts an ATM network laid out in a typical star topography. In the center of the star is the ATM switch. Numbered IP hosts are branched off of the switch as are links to other ATM switches and one ATM gateway box and adapter.

List of Representative Host Configurations
Network Interface Driver	IP Address	ARP Server	ARP Server Address	Gateway Address
Host H1
at0	128.10.1.3	Yes		128.10.1.5
Host H3
at0	128.10.1.1	No	ATM address of H1	128.10.1.5
Gateway
at0	128.10.1.5	No	ATM address of H1
at1	128.10.2.5	No	ATM address of H6
Host H4
at0	128.10.2.1	No	ATM address of H6	128.10.2.5
Host H6
at0	128.10.2.3	Yes		128.10.2.5

Note: Each subnet requires one and only one ARP server.

Because H3 recognizes that address 128.10.2.1 is not on its subnet, H3 consults H1 to resolve the default gateway IP address to an ATM address. H3 then places a call to the gateway. The gateway recognizes that the data is bound for the second subnet and consults H6 to successfully resolve the H4 IP address to an ATM address. Connections are then established between H3 and the gateway and between the gateway and H4.

Configuring an ATM Adapter

To configure your ATM adapter, use the Web-based System Manager, wsm, or the SMIT fast path smit chg_atm. Select an adapter name, then use the online help and multiple-choice lists to decide which changes to make for your configuration.

ATM Adapter Statistics

The atmstat command can be used for getting ATM adapter statistics. Using the atmstat command with the -r flag resets the statistics. The format of the command is atmstat DeviceName. This command returns the following sets of statistics:

Transmit Statistics

Packets:: This field contains the number of packets (or PDUs) transmitted.
Bytes:: This field contains the count of bytes transmitted. These are the user bytes. The ATM overhead, for example, ATM cell header, and AAL 5 PDU trailer, are excluded.
Interrupts:: This field is not used.
Transmit Errors:: This field contains the number of transmit errors for this device.
Packets Dropped:: This field contains the number of Transmit Packets that were dropped, for instance, because of an out of buffers condition.
Max Packets on S/W Transmit Queue:: This field does not apply to ATM.
S/W Transmit Queue Overflow:: This field does not apply to ATM.
Current S/W + H/W Transmit Queue Length:: This field contains the current transmit queue length.
Cells Transmitted:: This field contains the number of cells transmitted by this device.
Out of Xmit Buffers:: This field contains the number of packets dropped because of out of xmit buffers condition.
Current HW Transmit Queue Length:: This field contains the current number of transmit packets on the hardware queue.
Current SW Transmit Queue Length:: This field does not apply to ATM.

Receive Statistics

Packets:: This field contains the number of packets (or PDUs) received.
Bytes:: This field contains the count of bytes received. These are the user bytes. The ATM overhead, for example, ATM cell header and AAL 5 PDU trailer are excluded.
Interrupts:: This field contains the number of Interrupts taken by the system for the adapter-to-system indications. Some of the events that cause these interrupts are packet received, transmit done indication, and so on.
Receive Errors:: This field contains the number of receive errors for this device.
Packets Dropped:: This field contains the number of received packets dropped, for instance, due to out of buffers condition.
Bad Packets:: This field does not apply to ATM.
Cells Received:: This field contains the number of cells received by this device.
Out of Rcv Buffers:: This field contains the number of packets dropped because of out of receive buffers condition.
CRC Errors:: This field contains the number of received packets that encountered CRC errors.
Packets Too Long:: This field contains the number of received packets that exceeded the maximum PDU size.
Incomplete Packets:: This field contains the number of incomplete received packets.
Cells Dropped:: This field contains the number of dropped cells. Cells could be dropped for a number of reasons, such as bad header error control (HEC), out of buffer condition, and so on.

General Statistics

No mbuf Errors:: This field contains the number of mbuf requests that were denied.
Adapter Loss of Signals:: This field contains the number of times the adapter encountered loss of signal.
Adapter Reset Count:: This field contains the number of times the adapter has been reset.
Driver Flags: Up Running Simplex: This field contains the neighborhood discovery daemon (NDD) flags.
Virtual Connections in use:: This field contains the number of virtual connections that are currently allocated or in use.
Max Virtual Connections in use:: This field contains the maximum number of virtual connections allocated since the last reset of the statistics.
Virtual Connections Overflow:: This field contains the number of allocate virtual connections requests that have been denied.
SVC UNI Version:: This field contains the current UNI version of the signaling protocol in use.

Additional Microchannel ATM Statistics

Using the atmstat command with the -d flag provides detailed statistics.

Packets Dropped - No small direct memory access (DMA) buffer:: This field contains the number of received packets dropped because the adapter did not have small system buffers for DMA.
Packets Dropped - No medium DMA buffer:: This field contains the number of received packets dropped because the adapter did not have medium system buffers for DMA.
Packets Dropped - No large DMA buffer:: This field contains the number of received packets dropped because the adapter did not have large system buffers for DMA.
Receive Aborted - No Adapter Receive buffer:: This field contains the number of received packets aborted because the adapter did not have on-card receive buffers.
Transmit Aborted - No small DMA buffer:: This field contains the number of transmit packets dropped because of the lack of small system buffers for DMA.
Transmit Aborted - No medium DMA buffer:: This field contains the number of transmit packets dropped because of the lack of medium system buffers for DMA.
Transmit Aborted - No large DMA buffer:: This field contains the number of transmit packets dropped because of the lack of large system buffers for DMA.
Transmit Aborted - No MTB DMA buffer:: This field contains the number of transmit packets dropped because of the lack of large system buffers for DMA.
Transmit Aborted - No Adapter Transmit buffer:: This field contains the number of transmit packets dropped because of the lack of adapter on-card transmit buffers.
Max Hardware Transmit Queue Length:: This field contains the maximum number of transmit packets queued in the hardware queue.
Small Mbufs in Use:: This field contains the number of small mbufs currently in use. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
Medium Mbufs in Use:: This field contains the number of medium mbufs currently in use. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
Large Mbufs in Use:: This field contains the number of large mbufs currently in use. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
Huge Mbufs in Use:: This field contains the number of huge mbufs currently in use. The adapter device driver allocates these buffers according to the configuration information provided by the system administrators. This information can be used to tune the configuration information.
MTB Mbufs in Use:: This field contains the number of MTB mbufs currently in use. The adapter device driver allocates these buffers according to the configuration information provided by the system administrators. This information can be used to tune the configuration information.
Max Small Mbufs in Use:: This field contains the maximum number of small mbufs that have been used. The adapter device driver allocates these buffers according to the configuration information provided by the system administrators. This information can be used to tune the configuration information.
Max Medium Mbufs in Use:: This field contains the maximum number of medium mbufs that have been used. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
Max Large Mbufs in Use:: This field contains the maximum number of large mbufs that have been used. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
Max Huge Mbufs in Use:: This field contains the maximum number of huge mbufs that have been used. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
MTB Mbufs in Use:: This field contains the maximum number of MTB mbufs that have been used. The adapter device driver allocates these buffers according to the configuration information provided by system administrators. This information can be used to tune the configuration information.
Small Mbufs overflow:: This field contains the number of times that a small mbuf could not be allocated. This information can be used to tune the configuration information.
Medium Mbufs overflow:: This field contains the number of times that a medium mbuf could not be allocated. This information can be used to tune the configuration information.
Large Mbufs overflow:: This field contains the number of times that a large mbuf could not be allocated. This information can be used to tune the configuration information.
Huge Mbufs overflow:: This field contains the number of times that a huge mbuf could not be allocated. This information can be used to tune the configuration information.
MTB Mbufs overflow:: This field contains the number of times that an MTB mbuf could not be allocated. This information can be used to tune the configuration information.

PCI ATM Adapter Specific Statistics

Total 4K byte Receive Buffers: 768 Using: 512: This message contains the number of receive buffers allocated as well as the number that are currently in use.
Max 4K byte Receive Buffers limit: 1228 max_used: 514: This message contains the maximum number of receive buffers that can be allocated as well as the number that have been used since the adapter was last configured or opened.