Overview
Installing
and Configuring ANS
Structure of the ANS Build Tree
This file describes the Intel® Advanced Network Services (ANS) package, version 1.7.x, for Linux* systems. Intel ANS provides both teaming (including Multi-Vendor Teaming), and IEEE VLAN capabilities. For ANS support, this package version must be used with the Linux base driver version 2.0.x (or higher) for PRO/100 adapters and/or driver version 4.2.x (or higher) for PRO/1000 adapters.
This package was tested primarily on the Red Hat* Linux 2.4.7 kernel on both UP and SMP Intel processor-based systems.
Installation assumes all necessary build tools, kernel sources, and headers are in place and properly configured and that you are using the latest Intel base drivers found at http://support.intel.com. Base drivers provided in some distributions do not contain the necessary hooks for ANS.
Intel ANS may now be configured with a new command-line utility called PROCfg. This reporting and configuration tool reduces the command-line input required with the manual configuration method described in this document. At this time, PROCfg is only available for 32-bit architectures. For more information on using the PROCfg utility refer to procfg.htm. PROCfg is available on the Intel Customer Support website, http://support.intel.com.
This package supports the following teaming modes:
Adapter Fault Tolerance ("AFT"). Default mode when a team is created and the mode is not specified. A secondary adapter in the team automatically takes over on any failure of the primary connection (cable, adapter or port). Only one network adapter is active at a time. After a failover, if the connection to the primary adapter is restored, control passes automatically back to that primary adapter. May mix speed and duplex capabilities and settings. May use a hub or switch.
Adaptive Load Balancing ("ALB"). A team of 2-8 adapters. The primary adapter receives and transmits, and the transmission load is shared among all adapters in the team. Supports mixed speed/duplex settings among team members. Includes fault tolerance.
Intel Link Aggregation, Cisco*'s Fast EtherChannel* Technology or static 802.3ad ("FEC" or "FEC/LA/802.3ad: static"). A team of 2-8 10/100 adapters which simultaneously receive and transmit data. Includes fault tolerance and load balancing. Requires a switch that supports Intel Link Aggregation, Cisco*'s FEC or static 802.3ad. Must match speed/duplex settings on all team members. Spanning Tree Protocol must be turned off. Must match switch aggregation requirements.
Gigabit equivalent of FEC ("GEC" or "GEC/LA/802.3ad: static"). Requires PRO/1000 or equivalent adapters and a switch that supports Intel Link Aggregation, Cisco*'s GEC or static 802.3ad. Other requirements are similar to FEC.
IEEE 802.3ad: dynamic ("802.3ad"). This mode is the IEEE standard for the technology incorporated into Cisco's FEC method. ANS support for dynamic 802.3ad teaming is similar to that for FEC and GEC teaming. In addition to the benefits of Link Aggregation, IEEE 802.3ad Dynamic Link Aggregation generally offers the following advantages: Automatic Configuration, Rapid Configuration and Reconfiguration, and Deterministic Behavior. It consists of a team of 2-8 adapters that simultaneously receive and transmit data. Aggregation groups include only members with the same speed and full duplex. Includes fault tolerance and load balancing. Requires a switch that supports the IEEE 802.3ad dynamic standard, or at least Intel Link Aggregation or Cisco's FEC. In case one of the latter is used, the team will perform in 802.3ad static mode. Spanning Tree Protocol must be turned off. Must match switch aggregation requirements.
NONE. A team of one Intel adapter that adds VLAN support on top of one existing interface. Does not require an Intel server adapter. VLAN-enabled must be set.
 |
NOTES: The Intel ANS driver requires at least one Intel PRO/100 or PRO/1000 server adapter to form a team. |
| For AFT and ALB, you may specify which adapters in a team you want to use as the primary and secondary adapters. The primary one is the main adapter used in a team. For AFT, the primary one is the only adapter used until a failure occurs. For ALB, the primary one is the adapter that both transmits and receives. In a team of more than two adapters, the secondary adapter fill's the primary's role if the primary adapter fails. Also, fail back will not occur unless the primary has been specified. If Priority is not specified, the adapter with the highest supported speed is chosen. If all adapters support the same speed, the last one added is the primary adapter. |
Multi-Vendor Teaming (MVT) adds the capability to include adapters from selected vendors in a team (AFT, ALB, FEC, GEC, 802.3ad). For 802.3ad, the team performs in static 802.3ad mode. MVT cannot be used with VLANs.
In order to activate MVT, you must have at least one Intel server adapter in the team that is designated as the primary adapter. Adapters from other vendors must be properly loaded. Use the same command line parameters for building the team as with all Intel adapters. All members in an MVT operate on a common feature set.
|
NOTE: Use the manual installation method when installing and configuring MVT adapters. Use any network configuration tool to select the base driver for each adapter you wish to include in an ANS team. Make sure each driver is included as a loadable module and can be found in the appropriate location. Mark each adapter as "not active on boot" and remove any IP address assigned to it. This ensures that the driver is loaded and the adapter properly configured for use under ANS across reboots. |
IEEE 802.3ac Virtual LANs (VLANs) are based on the addition of a tag to the packet header. All equipment on the network that passes this traffic must be capable of handling the extra length. Settings on the adapter must match the VLAN settings on the switch. VLANs also add to load time and require additional memory resources. There is a limit of 64 VLANs per server; however, VLANs may be numbered from 1 to 4096.
When working with VLAN tagging, all the ports on the switch connected to the server should be configured to be tagged. In some Cisco switches it is possible to configure ports to be both tagged and untagged at the same time; although, this configuration will result in communication loss.
To install and compile the base driver package(s), see the readme included with those drivers. Make sure no other base drivers are loaded or compiled into the kernel for any of the Intel adapters (e.g., eepro, eepro100, eexpress, e100, e1000). Be sure the configuration files have been edited to alias the new driver(s). See the Example Setup section for an example file.
|
NOTE: Not for use with eepro100 (module removal may be necessary). |
Verify that all interfaces are working and that you have the base driver file(s), e100.o and/or e1000.o, located in:
For 2.2.x kernels: /lib/modules/<kernel_version>/net/
For 2.4.x kernels: /lib/modules/<kernel_version>/kernel/drivers/net/
The locations listed above may vary for certain Linux distributions. For exceptions, refer to ldistrib.htm.
Login as root.
Copy the iANS-x.x.x.tar.gz package file to a directory of your choice. "x.x.x" specifies the iANS version number.
Open the package file by running 'tar xzf iANS-x.x.x.tar.gz'. This creates a subdirectory for iANS-x.x.x under the current directory.
Enter the iANS-x.x.x directory. Choose one of these three methods for installation:
PROCfg Utility. This command-line interface provides a simplified configuration tool that reduces the amount of command-line input. At this time, it is only available for 32-bit architectures. For more information on using the PROCfg utility refer to procfg.htm. PROCfg is available on the Intel Customer Support website, http://support.intel.com.
Scripted. Suggested only for the following distributions:
Red Hat (6.x, 7, 7.1, 7.2) SuSE* (6.4, 7.1) Caldera OpenLinux* (2.2, 3.1) TurboLinux* (6.0 server, 6.1 server)
Manual (suggested for users with command-line experience)
ANS configuration and topology may be changed, with a few limitations. After commit, you may change member priorities, add or remove team members, add or remove virtual adapters, or remove a team entirely. You may not:
Change the team mode. Remove or add VLAN support. Remove a team's last virtual adapter. Remove the last member of a team with open virtual adapters.
If a team is left with no server adapter or with no members at all, it goes into an idle state until members are added. During such an idle period, resources used by the team are not freed.
For instructions on modifying ANS, see the instructions in either the scripted or manual configuration methods.
Removal of the ANS package includes:
|
|
Bringing down all virtual adapter interfaces. |
| |
Removing the ANS configuration from future reboots. |
| |
Removing the ANS module. |
|
Removing the files from the system. |
For instructions on removing ANS, see the instructions in either the scripted or manual configuration methods.
If ANS is already installed on your system and you wish to either modify or remove the package, refer to one of the following subsections under this heading:
For ease of configuration Intel supplies interactive scripts for the Linux distributions listed above. ANS sets up four directories named Red Hat, SuSE, Caldera, and Turbo to match each of these distributions under 'iANS-x.x.x/install_scripts/'.
To use the install script first follow the instructions in Installing and Configuring ANS. Then, enter:
iANS-x.x.x/install_scripts/[dist_name]
./INSTALL
SMP/UP and MODVERSIONS are determined by 'INSTALL'. If there is a mismatch between the configuration for the kernel source tree and the running kernel, a warning is displayed. Run 'INSTALL options' to see extra compilation switches that allow overriding the kernel source tree settings.
Follow the instructions displayed on the console. You must answer the following questions during the process:
"Would you like the configuration utility to remove all network interfaces automatically?" The installation process must bring down the network interfaces related to Intel adapters in order to unload the old Intel drivers and replace them with the new ones.
Select Yes (default) to remove all network interfaces, including other brands. Select No to choose each interface.
"Enter team name, (Enter) to end." You must enter either a team name (no longer than 15 characters, often ends in an instance number) or press <Enter> to quit adding teams. If you choose to quit, skip to Step 6.
"Would you like to have VLANs in team (name)?" Select Yes to create a VLAN-based team.
"Select members for team (name)" lists all applicable interfaces. Type the number of the adapter to add to the team. Select the role (none, primary or secondary). Repeat this for each adapter. Enter 0 when done.
"Enter virtual adapter name for team (name)." The virtual adapter name can be anything, except 'ethx', as long as it ends with an instance suffix and does not exceed 15 characters in length.
"Enter VLAN id for virtual adapter (name)." The VLAN id is for VLAN-based teams only. Enter a number from 1 to 4096 to specify the V LAN.
"Enter boot protocol (none, bootp, dhcp, or other)." Choose none to manually enter the IP address and netmask. Choose bootp or DHCP to enter the IP address and netmask from another machine. Or, choose other to enter the protocol name (such as IP).
|
NOTE: This information is NOT saved until you completely configure the team and confirm your selection. |
"Select Teaming Mode for Team (name) - AFT | ALB | FEC |
GEC | 802.3ad | NONE."
For descriptions, see the Overview section of this file. For Intel Link Aggregation, use
FEC or GEC.
 |
WARNING: Choosing NONE still creates a team. |
"Are you sure?" Choose Yes to commit to the chosen team. Choose No to return to step 3.
After committing to the team, you may make another team or press <Enter> to proceed.
"Would you like to save this configuration and load it on boot?" Choose Yes (default) to merge the ANS initialization script into the boot time sequence. Choose the run levels that should automatically load the ANS module. It is recommended that you select the same level as the network boot, usually 2,3,4, or 5. If you do not want to save the configuration for boot, select No.
"Would you like to start the network now?" Select Yes to activate the team. Choosing No should still show that installation and configuration completed successfully. ANS starts at reboot if the settings were saved in step 6.
To modify the ANS configuration, follow these instructions:
Manually modify the ANS topology description file:
/etc/ians/ians.conf
Call the ANS initialization script with the restart parameter:
/etc/rc.d/init.d/ans restart
To reactivate, run the network service initialization script with the restart parameter:
/etc/rc.d/init.d/network restart
Follow these instructions to remove ANS:
Make sure you are logged in as root.
Enter the iANS-x.x.x/install_scripts/[distribution_name] directory.
Run 'UNINSTALL'.
The rest of the process is interactive. Follow the instructions displayed on the console.
Prompts:
"Restore member adapters to original role (stand alone)." Answer 'Yes' for each adapter you wish to continue using stand alone mode. This configuration is made using Intel's e100(0).o driver; although, you may restore other drivers.
Removal of the ANS module from memory does not down the adapters running in stand alone mode. The script removes all files, scripts, and links created except '/etc/ians/ians.conf'.
For the new settings to take effect, you must restart the network service by either rebooting the system or running the network init script with the restart parameter:
/etc/rc.d/init.d/network restart
|
NOTE: If you used the automatic INSTALL method for configuring ANS note that lines were added to the '/etc/modules.conf' file. The UNINSTALL script does not remove these lines. If you want to remove them, you must do so manually. |
If ANS is already installed on your system and you wish to either modify or remove the package, refer to one of the following subsections under this heading:
To install and configure ANS, first complete the instructions in Installing and Configuring ANS. Then, follow these steps:
Enter the iANS-x.x.x/src directory, where "x.x.x" is the iANS version number. Run 'make' to compile the module.
|
NOTE: SMP/UP and MODVERSIONS are determined by 'make'. If there is a mismatch between the configuration for the kernel source tree and the running kernel, a warning is displayed. Run 'make options' to list the options available for overriding the source tree settings. Options include SMP, MODVERSIONS, and DEBUG (not listed). DEBUG values are 0 (debugging is off) and 1 (debugging is on). The default is 0. |
Run 'make install'.
|
NOTE: The default file locations are determined by 'make'. If there is a mismatch between the configuration for the kernel source tree and the running kernel, a warning is displayed. Either reboot to a kernel that matches the kernel source tree or set the symbolic link, usr/src/linux, to point to the kernel source tree that matches the running kernel. |
For a list of files and their default locations, see the Structure of the ANS Build Tree section in this file.
After the required base drivers are loaded, down any Intel adapters that are to be added to a team. If they already have an IP address, remove the address by running 'ifconfig ethx 0.0.0.0'.
ifconfig - a
- lists all current network devices ifconfig ethx down
- downs adapters so ANS may use them
|
NOTE: If you are not using Multi-Vendor Teaming, leave other vendor's Ethernet adapters configured, as well as any Intel adapters that are not added to teams. This ensures that the network device name assignment remains consistent after rebooting. |
Load the ANS module by running 'insmod ians'. In the following command lines, optional items are placed in [ ]. All VLAN settings are shown as optional.
To create teams and VLANs use the command line parameters listed in the Available Commands, General Configuration section.
Check the team status:
ianscfg -s
At this point the team is now active. You should be able to send and receive packets via the virtual adapter.
If you want the new configurations to take affect after each reboot, follow the instructions in Saving ANS Across Reboots.
To modify the ANS configuration, follow these instructions:
Take down the virtual adapter:
ifconfig <vadapter_name> down
Remove the ANS module:
rmmod ians
Create or modify the ANS topology file to match the new desired topology. Default location: /etc/ians/ians.conf. Or, create a script containing a set of calls to ianscfg that generates the desired topology.
Load the ANS module:
insmod ians
Run the script created in step 3, or run the following command, where <filename> is the new topology file:
ianscfg -r [-f <filename>]
Reset each created virtual adapter:
ifconfig <vadapter_name> <IPaddr> [netmask <NETMASK>] [broadcast <BROADCAST>]
Save the new configuration:
Create an interface configuration file for each new virtual adapter:ianscfg -w [-f <filename>]
The files would normally reside in '/etc/sysconfig/network-scripts/' The files would normally reside in '/etc/sysconfig/network-scripts/' Sample file content: DEVICE="<vadapter_name>"
BOOTPROTO="none"
ONBOOT="yes"
IPADDR="<IP_address>"
NETMASK="<netmask>"
If any unnecessary virtual adapter configuration file is left, remove it or modify it.
For any adapter that is desired as stand alone connection, you may continue to use the Intel base driver.
Follow these instructions to remove ANS:There are several stages to removing ANS:
Login as root.
Run 'ifconfig -a' to see every virtual adapter configured in the system.
For each virtual adapter, run 'ifconfig <vadapter_name> down'.
Remove the ANS module by running 'rmmod ians'.
In each virtual adapter config file, under /etc/sysconfig/network-scripts/, either modify the ifcfg-<vadapter_name> to contain the line 'ONBOOT="no"' instead of 'ONBOOT="yes"' or remove the file completely.
For each config file for stand alone adapters, under /etc/sysconfig/network-scripts/, modify the file ifcfg-ethx to contain the line 'ONBOOT="yes"' instead of 'ONBOOT="no"'.
You may leave the Intel base driver or you may edit the modules configuration file, '/etc/modules.conf', to restore its original form. To do this, remove the lines containing e100 or e1000 as the drivers that were added and restore every line that was commented out.
|
NOTE: For some 2.2.x kernel systems, the modules configuration file is named 'conf.modules'. |
Remove the ANS initialization script and all the links to it (e.g., '/etc/rc.d/init.d/ans' and '/etc/rc.d/rc.3/S09ans').
From the iANS-x.x.x/src directory, run 'make uninstall' to remove all the files.
The manual configuration method requires that you perform additional steps in order to save the new ANS configuration across reboots. After configuring the Ethernet and ANS drivers, follow these steps:
|
NOTE: The examples provided might vary for different Linux distributions. |
Save the ANS configuration to a file by running:
ianscfg -w [-f <file_name>]
The default is '/etc/ians/ians.conf'.
Make sure that the adapters that were selected as members of the team(s) are NOT configured on boot. To do so, change "ONBOOT=yes" to "ONBOOT=no" in every matching '/etc/sysconfig/network-scripts/ifcfg-ethx' file.
Set each virtual adapter to configure on boot by creating a config file, '/etc/sysconfig/network-scripts/ifcfg-<vadapter_name>'. The file must look something like this:
DEVICE="<vadapter_name>"
BOOTPROTO="none"
ONBOOT="yes"
IPADDR="<IP_address>"
NETMASK="<netmask>"
Create an init script that loads the e100.o, e1000.o (if needed) and ians.o modules and configures the ANS topology. You may use the sample script file by copying 'iANS-x.x.x/install/<Linux_distribution>/ans' to '/etc/rc.d/init.d/ans'. Configuration is read from '/etc/ians/ians.conf' by default. If you wish to use any other file, modify the script.
Add the ANS script to the init process for the appropriate run levels. Run 'chkconfig --level <run levels> ans on', where <run levels> is a set of run level where ANS starts. For example, 345.
|
NOTE: This script must be invoked BEFORE the system starts the network services. For example, if your network init script for run level 3 is '/etc/rc.d/rc3.d/S10network', create a symbolic link '/etc/rc.d/rc3.d/S09ans' that points to '/etc/rc.d/init.d/ans'. ANS service is automatically stopped when the network service is stopped. |
The following options describe common ANS commands:
|
NOTE: If you are using Multi-Vendor Teaming, do not use the "-V" or "-i" options. These flags are specific to VLANs, which are not supported with Multi-Vendor Teaming. |
|
|
Kernel dependant source files (src/*.[ch], src/lib/*.[ch]) |
| |
Kernel independent precompiled object file (bin/[ARCH]/ians_core.o) |
| |
Makefiles |
| |
ANS configuration utility (bin/[ARCH]/ianscfg) |
| |
ANS general utility daemon (bin/[ARCH]/iansd) |
| |
Installation, uninstall and configuration scripts |
|
NOTE: [ARCH] is either "ia32" or "ia64". |
By default, the components are distributed as follows:
- For 2.2.x kernels: /lib/modules/[KERNEL_VERSION]/net/ians.o
- For 2.4.x kernels: /lib/modules/[KERNEL_VERSION]/kernel/drivers/net/ians.o
- /usr/sbin/ianscfg
- /usr/sbin/iansd
- For Red Hat 6.x: /usr/man/man1/ianscfg.1.gz
- For Red Hat 7, 7.1, 7.2: /usr/share/man/man1/ianscfg.1.gz
#alias eth0 eepro100
#alias eth1 eepro100
#alias eth2 eepro100
alias eth0 e100
alias eth1 e100
alias eth2 e100
alias parport_lowlevel parport_pc
alias scsi_hostadapter aic7xxx
insmod e100
insmod ians
ianscfg -a -t team1 -M AFT -V
ianscfg -at team1 -m eth0 -p primary
ianscfg -at team1 -m eth1 -p secondary
ianscfg -at team1 -v vadapt1 -i 10
ianscfg -at team1 -v vadapt2 -i 15
ianscfg -c team1
ianscfg -s
ifconfig vadapt1 192.168.1.1 netmask 255.255.255.0
ifconfig vadapt2 192.168.2.1 netmask 255.255.255.0
|
NOTE: For distribution-specific issues, see ldistrib.htm. |
If an adapter has been running standalone with an assigned IP address, you must configure the IP address to 0.0.0.0 before adding it to a team:
ifconfig ethx {0.0.0.0}
Always bind the protocol to the virtual adapter and NOT to the physical adapter for adapters that are in a team. This applies to standalone adapters as well.
If you do not remove references to other possible base drivers in the configurations, you risk multiple interfaces and unpredictable outcomes for adapters you wish to use in the team.
If you receive an error message, "module not loaded," during configuration of the team, start over. To do so, enter:
./INSTALL veryclean all SMP=[0|1] MODVERSIONS=[0|1]
Choose 0 or 1 for SMP, and select MODVERSIONS to match the configuration for the current kernel tree.
|
NOTE: This error occurs when the target module does not match the kernel version and when install was done without parameters (found using './INSTALL' options). |
If, when adding the ANS script to the init process, you receive an error message, "service ans does not support chkconfig", manually create the symbolic link to the ANS file for each run level desired by entering:
ln -s /etc/rc.d/init.d/ans /etc/rc.d/rcX.d/S09ans
where X is the number of the desired run level.
If you are moving adapters in and out of teams with Jumbo Frames, check the MTU settings for Jumbo Frames.
If you receive the error message, "Intel Advanced Network Services (iANS) has determined you have selected network adapters that are not eligible to participate as members of an iANS team. Please install Intel Server Adapters in your system and re-install the advanced server features.", this indicates that either:
the team needs at least one Intel server adapter, or
the last chosen adapter is from another vendor and is not capable of participating in the team.
When trying to compile the driver by running make install, the following error may occur:
"Linux kernel source not configured - missing version.h"
To solve this issue, create the version.h file by going to the Linux source tree and doing a make include/linux/version.h.
Setting a non-Intel adapter to a priority higher than the current
primary adapter's priority can cause the system to lose its
stability and enter a mode where continuous failovers occur.
This type of failover tends to appear when certain switches have a
delayed recovery time after a link loss. If you are using a switch and
notice such behavior, avoid setting an MVT adapter as the primary adapter.
PCI Hot Plug Add/Remove.
When executing a PCI Hot Plug Remove operation on an adapter that is part of an ANS team, the adapter is automatically removed from the team.
When reactivating an adapter with a PCI Hot Plug Add operation, the new interface should be manually added back into the team by an ANS Hot Add operation.
For general information and support, go to the Intel support website at:
If an issue is identified with the released source code on the supported kernel with a supported adapter, email the specific information related to the issue to linux.nics@intel.com.
Copyright © 2002 Intel Corporation. Legal Information