In my first blog post about building out servers with ARM processors, I had mentioned that one could build a high-density scale-out server infrastructure by fitting 20 blades into a 3U chassis:

How would 20 ARM-based server nodes be able to fit into a 3U chassis? It has been done before, even with nodes that pull more than 15-20W each. Both Sun Microsystems and Compaq had a 3U blade server chassis that supported 20 blades (with one UltraSPARC IIe/IIi or one Pentium M processor, 1-2GB of RAM and one 2.5″ hard drive bay) and an Ethernet switch or pass-through module. One can use the same blade setup, use smaller and more efficient power supplies and cooling (as the need for power and cooling with be a lot less than 20W per blade), update the switch to support Gigabit downstream ports and 10Gbps Ethernet uplink ports, and reduce the chassis depth. I would even bet that one could find a way to fit 20 nodes in 2U of space without sacrificing any functionality or availability.

Well, HP has taken that idea and ratcheted it up to a very impressive scale. Project Moonshot crams 72 server nodes into a 2U half-width tray housing eighteen Calxeda EnergyCards and four external 10Gb/s XAUI ports. Four of those trays can slot into a 4U SL6500 chassis, for a total of 288 nodes (72 nodes per 1U). Each EnergyCard contains four EnergyCore processors with up to 4GB per processor and 4 SATA ports, all while drawing 25 watts. In turn, each EnergyCore processor comes in both two and four Cortex-A9 core configurations and has a high-throughput fabric switch built-in. The fabric switch provides multiplexed access to five 10Gb/s XAUI ports and six 1Gb/s SGMII ports, all of which is wrapped around by three 10GbE MAC ports. Each EnergyCore also provides five SATA ports, several PCIe controllers and an SD/eMMC controller (say, for booting an operating system).

While such an impressive setup may not immediately fit into common enterprise workloads, don’t be surprised to see these things popping up at places where companies need an enormous amount of light/moderate duty workloads that can be scaled out across thousands of threads; and, where the cost of power and cooling are at a premium. Both Ubuntu and Fedora can be used, though Windows Server 8 might be an option if Microsoft deems it to be worth the time and money.