[ Bottom of Page | Previous Page | Next Page | Contents | Index | Library Home |
Legal |
Search ]
System Management Guide: Communications and Networks
EtherChannel
EtherChannel is a network port aggregation technology that allows several
Ethernet adapters to be aggregated together to form a virtual Ethernet adapter.
The adapters that belong to an EtherChannel are cabled to the same EtherChannel-enabled
network switch, which must be manually configured to identify the ports that
belong to the EtherChannel.
The system sees the EtherChannel as any other Ethernet adapter. For this
reason, IP is configured over an EtherChannel adapter as over any Ethernet
adapter. Furthermore, all the adapters in the EtherChannel are automatically
configured with the same hardware (MAC) address, so they are treated by remote
systems as if they were one adapter.
EtherChannel's main benefit is that its interface has the aggregated network
bandwidth of all its adapters. Traffic is distributed across the adapters
in either the standard way (where the adapter over which the packets are sent
is chosen depending on the destination address) or on a round-robin basis
(where packets are sent evenly across all adapters). If an adapter fails,
the packets are automatically sent on the next available adapter in the EtherChannel
without disruption to existing user connections. Once the adapter failure has
been corrected, the adapter can be used for network traffic once again.
In AIX, users can configure multiple EtherChannels per system,
but it is required that all the links in one EtherChannel are attached to
a single switch. Because the EtherChannel cannot be spread across two switches,
the entire EtherChannel is lost if the switch is unplugged or fails. To solve
this problem, a new backup option added in AIX 5.2 keeps the service
running when the main EtherChannel fails. The backup and EtherChannel adapters
should be attached to different network switches. In the event that all of
the adapters in the EtherChannel fail, the IP and MAC addresses will be automatically
moved to the backup adapter. When any link in the EtherChannel is restored,
the service is moved back to the EtherChannel.
Network Interface Backup, a mode of operation for EtherChannel, protects
against a single point of Ethernet network failure. In Network Interface Backup
mode, only one adapter at a time is actively used for network traffic. The
EtherChannel tests the currently-active adapter and, optionally, the
network path to a user-specified node. When a failure is detected, the
MAC and IP addresses are moved to the next adapter, which will be used until
it fails. Network Interface Backup provides rapid detection and failover with
no disruption to user connections. Network Interface Backup was originally
implemented as a mode in the EtherChannel SMIT menu. In AIX 5.2, the
backup adapter provides the equivalent function, so the mode was eliminated
from the SMIT menu. To use network interface backup in AIX 5.2, see Configure Network Interface Backup.
Configuring EtherChannel
The following procedure will help you set up an EtherChannel.
Considerations
- You can have anywhere from two to eight Ethernet adapters per EtherChannel.
- You can configure multiple EtherChannels on a single system, but remember
that each EtherChannel constitutes an additional Ethernet interface. The no command option, ifsize, may need to be increased to include not only the Ethernet interfaces
for each adapter, but also any EtherChannels that are configured. The default ifsize is eight.
- You can use any supported Ethernet adapter in EtherChannel. However, the
Ethernet adapters must be connected to a switch that supports EtherChannel.
See the documentation that came with your switch to determine if it supports
EtherChannel.
- All adapters in the EtherChannel should be configured for the same speed
(10 Mbps, for example) and must be configured with the same mode (full duplex,
for example).
- The adapters that you plan to use for your EtherChannel must not have
an IP address configured on it before you start this procedure. Use the ifconfig command to unconfigure
your adapters. For example, ifconfig en5 detach would unconfigure adapter en5.
Configure an EtherChannel
- With root authority, type smit etherchannel at the command
line.
- Select Add an EtherChannel from the list and press
Enter.
- Select the primary Ethernet adapters that you want on your EtherChannel
and press Enter. If you are planning to use EtherChannel backup, do not select
the adapter that you plan to use for the backup at this point. The EtherChannel
backup option was added in AIX 5.2.
Note
The Available Network Adapters displays all Ethernet adapters. If you select
an Ethernet adapter that is already being used (has an interface defined),
you will get an error message. You first need to detach these interfaces if
you want to use them.
- Enter the information in the fields according to the following guidelines:
- EtherChannel Adapters: You should see all primary
adapters that you are using in your EtherChannel. You selected these adapters
in the previous step.
- Enable Alternate EtherChannel Address: This field
is optional. Setting this to yes will enable you to specify a MAC
address that you want the EtherChannel to use. If you set this option to no, the EtherChannel will use the MAC address of the first adapter.
- Alternate EtherChannel Address: If you set Enable Alternate EtherChannel Address to yes, specify the
MAC address that you want to use here. The address you specify must start
with 0x and be a 12-digit hexadecimal address.
- Enable Gigabit Ethernet Jumbo Frames: This field
is optional. In order to use this, your switch must support jumbo frames.
This will only work with Standard Ethernet, not IEEE 802.3. Set this to yes if you want to enable it.
- Mode: You can choose from the following modes:
- Standard: In this mode the EtherChannel uses the
destination IP address to choose which adapter it will send the packets out
on. The EtherChannel divides the last byte of the packet's destination IP
address by the number of adapters in the EtherChannel and uses the remainder
(using the "modulus" operator) to identify the outgoing link. For instance,
if the destination IP is 10.10.10.1, and there are 2 adapters in the EtherChannel,
(1 % 2) = 1, so the second adapter is used (the adapters are numbered
starting from 0). The adapters are numbered in the order they are listed in
the SMIT menu. For non-IP traffic (such as ARP), the last byte of the
destination MAC address is used to do the calculation. This mode will guarantee
packets are sent out over the EtherChannel in the order they were given to
it, but it may not make full use of the bandwidth. This is the default operation
mode.
- Round Robin: In this mode the EtherChannel will
rotate through the adapters, giving each adapter one packet before repeating.
The packets may be sent out in a slightly different order than they were given
to the EtherChannel, but it will make the best use of its bandwidth.
- Network Interface Backup: This option is available
in AIX 5.1 and AIX 4.3.3. In this mode, the channel will activate
only one adapter at a time. The intention is that the adapters are plugged
into different Ethernet switches, each of which is capable of getting to any
other machine on the subnet or network. When a problem is detected either
with the direct connection (or optionally through the inability to ping a
machine), the EtherChannel will deactivate the current adapter and activate
a backup adapter. This mode is the only one that makes use of the Internet Address to Ping, Number of Retries, and Retry Timeout fields.
Network Interface Backup Mode does
not exist as an explicit mode in AIX 5.2. To enable Network Interface
Backup Mode in AIX 5.2, you must configure one adapter in the main EtherChannel
and a backup adapter. For more information, see Configure Network Interface Backup.
- Backup Adapter: This field is optional. Enter the
adapter that you want to use as your EtherChannel backup. EtherChannel backup
is available in AIX 5.2.
- Internet Address to Ping: This field is optional
and is only available if you are running Network Interface
Backup mode. The EtherChannel will ping the IP address that you specify
here. If the EtherChannel is unable to ping this address for the Number of Retries times in Retry Timeout intervals,
the EtherChannel will switch adapters.
- Number of Retries: Enter the number of ping response
failures that are allowed before the EtherChannel switches adapters. The default
is three. This field is optional and valid only if you have set an Internet
Address to Ping.
- Retry Timeout: Enter the number of seconds between
the times when the EtherChannel will ping the Internet Address
to Ping. The default is one second. This field is optional and valid
only is you have set an Internet Address to Ping.
- Press Enter after changing the desired fields to create the EtherChannel.
- Configure IP over the newly created EtherChannel device by typing smit chinet at the command line.
- Select your new EtherChannel interface from the list.
- Fill in all the required fields and press Enter.
Configure Network Interface Backup
Network Interface Backup protects against a single point of network failure
by providing failure detection and failover with no disruption to user connections.
When operating in this mode, only one adapter is active at any given time.
If the active adapter fails, the next adapter in the EtherChannel will be
used for all traffic. When operating in Network Interface Backup mode, it
is not necessary to connect to an EtherChannel-enabled switch.
The Network Interface Backup setup is most effective when the adapters
are connected to different network switches, as this provides greater redundancy
than connecting all adapters to one switch. When connecting to different switches,
make sure there is a connection between the switches. This provides failover
capabilities from one adapter to another by ensuring that there is always
a route to the currently-active adapter.
In releases previous to AIX 5.2, Network Interface Backup mode was
implemented as an explicit mode of operation in the EtherChannel SMIT menu.
In AIX 5.2, however, the backup adapter functionality provides the equivalent
behavior, so the mode was eliminated from the SMIT menu.
AIX 5.2 additionally provides priority, meaning that the adapter
configured in the primary EtherChannel will be used over the backup adapter.
As long as the primary adapter is functional, it will be used. This contrasts
from the behavior of Network Interface Backup mode, where the backup adapter
was used until it also failed, regardless of whether the primary adapter had
already recovered.
While operating in Interface Backup Mode, it is also possible to configure
the EtherChannel to detect link failure and network unreachability. To do
this, specify the IP address of a remote host where connectivity should always
be present. The EtherChannel will periodically ping this host to determine
whether there is still a network path to it. If a specified number of ping
attempts go unanswered, the EtherChannel will fail over to the next backup
adapter in the hope that there is a network path to the remote host through
the next adapter. In this setup, not only should every adapter be connected
to a different switch, but each switch should also have a different route
to the host that is pinged.
This ping feature is only available in Network Interface Backup mode. However,
in AIX 5.2, if the ping feature is enabled and a failover due to ping
failure has occurred, the EtherChannel will not fail back to the primary adapter.
The backup adapter will be the active channel as long as it is working because
there is no way to know when the route to the pinged host will become reachable
from the primary adapter. If a failure is detected while the backup adapter
is active (that is, if either the ping attempts fail from the backup adapter
or if the backup adapter itself fails), the EtherChannel will then failover
to the primary adapter. If the failover occurred because the primary adapter
failed, the EtherChannel will then come back to the primary adapter as soon
it has come back up.
To configure Network Interface Backup in AIX 5.2, see Configure Network Interface Backup in AIX 5.2.
To configure Network Interface Backup in previous versions of AIX,
see Configure Network Interface Backup in previous AIX versions
Configure Network Interface Backup in AIX 5.2
- With root authority, type smit etherchannel on the command
line.
- Select Add an EtherChannel from the list and press
Enter.
- Select the primary Ethernet adapter and press Enter. This is the adapter
that will be used until it fails.
Note
The
Available
Network Adapters displays all Ethernet adapters. If you select an Ethernet
adapter that is already being used, you will get an error message and will
need to detach this interface before you can use it. See the
ifconfig command for information on how to detach an
interface.
- Enter the information in the fields according to the following guidelines:
- EtherChannel Adapters: You should see the primary
adapter you selected in the previous step.
- Enable Alternate EtherChannel Address: This field
is optional. Setting this to yes will enable you to specify a MAC
address that you want the EtherChannel to use. If you set this option to no, the EtherChannel will use the MAC address of the primary adapter.
- Alternate EtherChannel Address: If you set Enable Alternate EtherChannel Address to yes, specify the
MAC address that you want to use here. The address you specify must start
with 0x and be a 12-digit hexadecimal address.
- Enable Gigabit Ethernet Jumbo Frames: This field
is optional. In order to use this, your switch must support jumbo frames.
This will only work with Standard Ethernet, not IEEE 802.3. Set this to yes if you want to use it.
- Mode: It is irrelevant which mode of operation you
select because there is only one adapter in the main EtherChannel. All packets
will be sent over that adapter until it fails. There is no netif_backup mode because that mode can be emulated using a backup adapter.
- Backup Adapter: Enter the adapter that you want
to be your backup adapter. After a failover, this adapter will be used until
the primary adapter recovers. It is recommended to use the preferred adapter
as the primary adapter.
- Internet Address to Ping: The field is optional.
The EtherChannel will ping the IP address that you specify here. If the EtherChannel
is unable to ping this address for Number of Retries time
in Retry Timeout intervals, the EtherChannel will switch
adapters.
- Number of Retries: Enter the number of ping response
failures that are allowed before the EtherChannel switches adapters. The default
is three. This field is optional and valid only if you have set an Internet Address to Ping.
- Retry Timeout: Enter the number of seconds between
the times when the EtherChannel will ping the Internet Address
to Ping. The default is one second. This field is optional and valid
only if you have set an Internet Address to Ping.
- Press Enter after changing the desired fields to create the EtherChannel.
- Configure IP over the new interface by typing smit chinet at
the command line.
- Select your new EtherChannel interface from the list.
- Fill in all the required fields and press Enter.
Configure Network Interface Backup in previous AIX versions
- With root authority, type smit etherchannel on the command
line.
- Select Add an Etherchannel and press Enter.
- Select the adapters that you want to include in the EtherChannel, both
primary and secondary.
Note
The
Available Network
Adapters displays all Ethernet adapters. If you select an Ethernet adapter
that is already being used, you will get an error message. You first need
to detach that interface by using the
ifconfig command.
- Enter the information in the fields according to the following guidelines:
- Etherchannel adapters: You should see the adapters
that you selected in the previous step.
- Enable ALTERNATE ETHERCHANNEL address: This field
is optional. Setting this to yes will enable you to specify a MAC
address that you want the EtherChannel to use. If you set this option to no, the EtherChannel will use the MAC address of the first adapter specified.
- ALTERNATE ETHERCHANNEL address: If you set Enable ALTERNATE ETHERCHANNEL address to yes, specify the
MAC address that you want the EtherChannel to use here. The address you specify
must start with 0x and be a 12-digit hexadecimal value.
- Mode: Select netif_backup.
- Enable GIGABIT ETHERNET JUMBO frames: This field
is optional. In order to use this, your switch must support jumbo frames.
This will only work on Standard Ethernet, not IEEE 802.3. Set this to yes if you want to enable it.
- Internet Address to Ping: This field is optional.
The EtherChannel will ping the IP address you specify here. If the EtherChannel
is unable to ping this address for Number of Retries times
in Retry Timeout intervals, the EtherChannel will switch
adapters.
- Number of Retries: Enter the number of ping response
failures that are allowed before the EtherChannel switches adapters. The default
is three. This field is optional and valid only if you set an Internet Address to Ping.
- Retry Timeout: Enter the number of seconds between
the times when the EtherChannel will ping the Internet Address
to Ping. The default is one second. This field is optional and valid
only if you have set and Internet Address to Ping.
- Press Enter after changing the desired fields to create the EtherChannel.
- Configure IP over the new interface by typing smit chinet at
the command line.
- Select your new EtherChannel interface from the list.
- Fill in all required fields and press Enter.
Managing an EtherChannel
This section will tell you how to perform the following tasks:
List EtherChannels
- On the command line, type smit etherchannel.
- Select List All EtherChannels and press Enter.
Change the Alternate Address
This enables you to specify a MAC address for your EtherChannel.
- Detach the EtherChannel by typing ifconfig interface detach, where interface is
your EtherChannel's interface.
- On the command line, type smit etherchannel.
- Select Change / Show Characteristics of an EtherChannel and press Enter.
- If you have multiple EtherChannels, select the EtherChannel for which
you want to create an alternate address.
- Change the value in Enable Alternate EtherChannel Address to yes.
- Enter the alternate address in the Alternate EtherChannel
Address field. The address must start with 0x and be a 12-digit
hexadecimal address.
- Press Enter to complete the process.
Add, remove, or change adapters in an EtherChannel
- Detach the EtherChannel by typing ifconfig interface detach, where interface is
your EtherChannel's interface.
- On the command line type, smit etherchannel.
- Select Change / Show Characteristics of an EtherChannel and press Enter.
- Select the EtherChannel that you want to modify.
- Select the primary adapters that you want in your EtherChannel and press
Enter. If you are using EtherChannel backup, do not select that adapter here.
- Fill in the necessary fields and press Enter.
Remove an EtherChannel
- Detach the EtherChannel by typing ifconfig interface detach, where interface is
your EtherChannel's interface.
- On the command line type smit etherchannel.
- Select Remove an EtherChannel and press Enter.
- Select the EtherChannel that you want to remove and press Enter.
Configure or remove a backup adapter on an existing EtherChannel
The following procedure configures or removes a backup adapter on an EtherChannel.
This option is available only in AIX 5.2.
- Detach the EtherChannel by typing ifconfig interface detach, where interface is
your EtherChannel's interface.
- On the command line, type smit etherchannel.
- Select Change / Show Characteristics of an EtherChannel.
- Select the EtherChannel that you are adding or modifying the backup adapter
on.
- Enter the adapter that you want to use as your backup adapter in the Backup Adapter field, or select NONE if
you wish to stop using the backup adapter.
Troubleshooting EtherChannel
If you are having trouble with your EtherChannel, you can use tcpdump and iptrace to troubleshoot the EtherChannel. The trace hook id for the
transmission packets is 2FA and for other events is 2FB. You cannot trace
receive packets on the EtherChannel as a whole, but you can trace each adapter's
receive trace hooks.
You can also use the entstat command to get the aggregate statistics of all the adapters in the
EtherChannel. For example, entstat ent7 will display the aggregate
statistics of ent7. Adding the -d flag will also display the statistics
of each adapter individually. For example, typing entstat -d ent7 will
show you the aggregate statistics of the EtherChannel as well as the statistics
of each individual adapter in the EtherChannel.
Note
In the
General Statistics section, the number shown in
Adapter Reset Count is the number of failovers. In EtherChannel backup, coming back to
the main EtherChannel from the backup adapter is not counted as a failover.
Only failing over from the main channel to the backup is counted.
In the Number of Adapters field, the backup adapter is counted in the number
displayed.
If you notice that the failover time when you are using network interface
backup mode or EtherChannel backup is much greater than five seconds, you
should verify that your switch is not running the Spanning Tree Protocol (STP).
When the switch detects a change in its mapping of switch port to MAC address,
it runs the spanning tree algorithm to see if there are any loops in the network,
and this may take some time. Network Interface Backup and EtherChannel backup
may cause a change in the port to MAC address mapping, so disabling the algorithm
should make the failover faster.
Also, note that switch ports have a forwarding delay counter that determines
how soon after it is initialized each port should begin forwarding or sending
packets. For this reason, when the main channel is re-enabled, you may notice
that there is a delay of a few seconds before the connection is re-established,
whereas the failover to the backup adapter is much faster. You might want
to check the forwarding delay counter configured in your switch and make it
as small as possible so that coming back to the main channel occurs as fast
as possible.
Specifically, for the EtherChannel backup function to work correctly, the
forwarding delay counter must not be more than 10 seconds, or coming back
to the main EtherChannel might not work correctly. Setting the forwarding
delay counter to the lowest value allowed by the switch is recommended.
If you find that adapter failures are not triggering failovers (in other
words, if an adapter that is down is still being used by an EtherChannel to
send packets), you might want to check to see if your adapter card needs to
have link polling enabled to detect link failure. Some adapters cannot automatically
detect their link status (whether they are up or down). To detect said condition,
these adapters must enable a link polling mechanism, whereby a timer is started
that periodically verifies the status of the link. This link polling is disabled
by default because it decreases performance. For EtherChannel to work correctly
with these adapters, however, the link polling mechanism must be enabled on
each adapter before the EtherChannel is created.
Adapters that have a link polling mechanism have an ODM attribute called poll_link, which must be set to yes for the link
polling to be enabled. Before creating the EtherChannel, use the following
command on every adapter to be included in the channel:
chdev -l ent? -a poll_link=yes
Likewise, for the jumbo frames option to work properly, aside from enabling
the use_jumbo_frame attribute on the EtherChannel, you
must also enable jumbo frames on each adapter before creating the EtherChannel
using the following command:
chdev -l ent? -a jumbo_frames=yes
Note
In AIX 5.2, this is done automatically when the use_jumbo_frame attribute is set on the EtherChannel.
Remote dump is not supported over an EtherChannel.
[ Top of Page | Previous Page | Next Page | Contents | Index | Library Home |
Legal |
Search ]