Things We Learned about the Apple Xserve RAID

Every product has interesting little quirks, gotchas, and things that are nice about it that the manufacturer does not do justice to. This is a list of everything we have noted, that other people might be interested in, especially if you want to use these things in a large environment.

At the bottom of this document we also have some ideas for using the Xserve RAID in enterprise environments.

Apple has a two-part FAQ (one, two) on the Xserve RAID that you'll probably want to read first.

Our observations:

• The Apple Xserve RAID does not have GBICs on it. Each controller has a single female HSSDC2 connection that is capable of a 2 Gb connection (point-to-point or arbitrated loop). The copper cables that come with the Xserve RAID to connect it to the Apple Fibre Channel PCI Card have a male HSSDC2 connector on one end and an SFP tranceiver on the other end. The Xserve RAID has no optical fibre channel capabilities whatsoever.
  
  Because of this configuration, you either need to place the Xserve RAID near the hosts that will use it, or cable the Xserve RAID to a fibre channel hub or switch that can accept SFP devices.
  
  If you cable the Xserve RAID to a switch or hub you will be able to connect other hosts to it via optical/fiber cabling. We have done this and it works well -- we have shared our Xserve RAIDs to AIX and Linux hosts across our existing Brocade SAN.
  
  UPDATE: Apple now has optical GBICs on the new Xserve RAIDs, thereby enhancing the connectivity via fibre channel switches (and long distance cable runs).
  
  
• The Apple Xserve RAID is divided into 2 seven disk arrays. They each can be set up as RAID 0, 1, 3, 5, or JBOD. You cannot make a 14 disk array in hardware on the Xserve RAID.
  
  
• It takes nearly 24 hours to format a seven disk array in a RAID 5 configuration.
  
  
• Both of the Apple Xserve RAID's controllers can manage the array. However, each controller can only see one half of the drives. In order to see both seven-disk arrays you need to be cabled to both of the controllers.
  
  This means that if you lose connectivity to one of the controllers some of your storage is inaccessible, too. You can get around this with the built-in RAID in Mac OS X, Linux, Windows, etc., but you do lose half of your storage (turning your 2 TB array into 1 TB). You could probably use the native RAID software in your OS to turn a few Xserve RAIDs into an array of their own (stripe across multiple Xserve RAID units). Remember: much of your other storage, including the SCSI hardware inside your servers, is single-connected, too.
  
  
• You can build smaller arrays out of the Xserve RAID disks. Unless you are setting them up as JBOD, you will probably only get two arrays (3 & 4 disks) out of each side of the Xserve RAID. You can use LUN masking to present each array to one or more hosts (and only those hosts). Since you can only have two connections, one per side, this implies the use of a fibre channel switch.
  
  UPDATE: With the new Xserve RAID firmware you can "slice" the arrays with up to six slices, and allocate the slices to the hosts separately.
  
  
• The Apple Fibre Channel PCI Card is an LSI card that accepts SFP connectors, like GBICs. You can purchase optical GBICs from a variety of places if you wish to use these cards as an inexpensive alternatives to Emulex or Qlogic cards. We have not tested the performance of the Apple Fibre Channel PCI Card against the Qlogic or Emulex offerings, so there might be some differences there.
  
  LSI supplies drivers for other operating systems. Newer OSes, like Red Hat Enterprise Linux 3.0, have the LSI driver built in.
  
  
• The RAID Admin software from Apple is a Java application. You can run it on Windows or any platform that supports Java (we've tried Windows and Mac OS X). You can only download it from Apple in Macintosh DMG format, though, so if you get an Xserve RAID you might want to have a Mac OS X machine around, too. Because it's a Java application your Windows-loving coworkers can help you admin the array, too.
  
  
• When you purchase an Apple Xserve RAID you might want to get the AppleCare Service Part Kit for the Xserve RAID, especially if you are ordering more than one. Also get the 512MB RAID controller cache. It's $300 dollars, and is well worth it.
  

With all that said, there are some great potential uses for Apple Xserve RAID beyond the obvious (the obvious being connecting it directly to your server for storage):

• The Apple Xserve RAID is a great back end for storage virtualization products like those from MaXXan or Datacore. If you are not familiar with storage virtualization, these products basically take a lot of the features of disk arrays and put them in a controller/intermediate device. You allocate all of the storage to the virtualization controller and then it reshares the storage to other hosts. The controller usually has a ton of cache on it, and can handle mirroring data to another controller or set of disks. The virtualization controllers can also do snapshots of your storage, too.
  
  Many sites that are concerned with disaster recovery can use the Apple Xserve RAID as a storage device even if they cannot use it as primary storage. You can do snapshots and mirroring to the Xserve RAID arrays, and combined with a pair of fibre channel switches (with long wave GBICs) you can place the arrays in a remote building.
  
  With virtualization controllers you can migrate a host's storage seamlessly, too (the host never knows where its storage resides). You can use Xserve RAID arrays as a tier in your service offerings, or use them to migrate lower-intensity servers.
  
  
• Backup software, like IBM Tivoli Storage Manager, lets you create backup storage pools out of disk resources. TSM even lets you store one copy on the disk pool and make another copy to tape, in case something happens to the disk (or vice versa). Restores are fast and the whole operation is safe.
  
  Along these lines are HSM (Hierarchical Storage Management) products, where files & data that have not been used for a certain amount of time are migrated automatically to lower performance disk and/or tape. Classically the data has been moved to tape -- with Xserve RAID you can inexpensively migrate them to ATA disk.
  
  
• OpenFiler: this thing turns a server with storage into a NAS/iSCSI/CIFS/AFP monster.
  
  
• Clusters of machines: since the Xserve RAID is best at being a two-array device, just share the storage among many hosts. For Linux you need to run GFS, AIX can use GPFS, etc.