Eric Severance

Using my list of requirements, I set out to find the hardware for my new server. I was building this from scratch so at minimum my purchase list needed to include

1. hard drive storage
2. server case
3. motherboard, RAM & CPU

Hard drive storage
I decided to focus first on the requirements for the fileserver side of the project. Recall that I was planning for 16TB of storage space. At the time, the largest consumer hard drives were 2 TB. I also wanted to be able to support multiple drive failures and be able to replace the drives without shutting down the system. That meant I needed at least 10 hard drives.

When researching the hardware for this server I came across a good blog post from Adaptec about real life RAID reliability. That article compared the reliability of RAID-5 and RAID-6 arrays and showed that a RAID-6 array should last 172 times longer than a RAID-5 array. Reliability was important to me on this project, so I decided to go with RAID-6. The Adaptec article only considered enterprise grade drives. I planned to build this server with consumer grade drives. Therefore, as a precaution, I chose to add two extra drives as hot spares.

A RAID-6 array with 10 drives was likely to run slow. So I did some more searching and came across RAID-60. RAID-60 combines the redundancy of RAID-6 with the speed of striping found in RAID-0. However, to get 16 TB with RAID-60, and have two hot spares, I now needed 14 hard drives. Six drives for each of the two RAID-6 arrays and two hot spares.

I wanted to make sure the fileserver would run quickly so that I could stream video from it while MythTV was also recording new programs and all the virtual machines were also running. I thought running everything off one set of storage drives might be too much, so I decided to split the VM storage from the NAS storage. That meant adding additional drives. I had four 1TB drives from my previous NAS, so I decided to use them for storing the VM images.

That put the total number of hard drives needed at 18. This was shaping up to be quite a storage server! The next task was determining how to fit that many hard drives into a computer case.

Server Case
I wanted the server to be able to stay running while I replaced a failed drive, so I needed a case that accommodated hot swappable drive bays. I considered putting the drives into some six-bay external drive enclosures, but decided that would get too expensive and end up using more power then was needed. Plus, I could just see the cables getting disconnected between the external enclosures and the main CPU.

No tower-style case that I could find would hold that many drives, so I looked for rack mountable cases. To fit 18 drives, a 4U rack mount case was needed.

Motherboard, RAM & CPU
I wanted to be able to expand this system in the future, so when choosing the motherboard I focused on server boards that supported dual CPUs. My plan was to put the system together with one CPU, and if needed, add another CPU later. I also needed to find a motherboard with multiple network interfaces, and plenty of PCI-express slots for adding RAID cards. Since reliability was important to me, I focused only on motherboards that supported ECC RAM. Form factor wasn't a big issue for this system as it was being placed into a rack mount case with plenty of space.

For the CPU, I needed a processor that supported VT-d. VT-d processors support mapping cards plugged into PCIe slots directly into virtual machines. My goal was to create a virtual machine for the fileserver and map the RAID card directly into that VM.

Another goal of mine was to make the new server easy to administer. I didn't want to have to find a spare keyboard, mouse, and monitor and plug them all in when there was trouble. The solution, IPMI. A motherboard with IPMI would allow me to remotely control the keyboard, mouse, video and even attach a remote DVD-ROM to perform an OS install. It is basically a built-in KVM over IP. I can even remotely reset the computer using IPMI.

Parts list
I ended up purchasing the following components for this system

• 14 x 2 TB hard drives (5 x Hitachi HDS72202, 3 x SAMSUNG HD203WI, 6 x WDC WD20EADS)
• Norco RPC-4220 4U rackmount case with 20 hot swappable drive bays
• Supermicro X8DTi-F motherboard with 3 PCIe 8x slots and IPMI
• 24Gb ECC Registered DDR3 1066 RAM
• Intel Xeon E5506 Nehalem-EP 2.14 GHz processor