A Rack of ARMs: 16 cores in 1U

If you have been following my blog recently, you will probably have seen my advocacy of using servers with ARM processors in large scale-out environments due to their small form factor and very low power consumption. Both of which allows a single rack to host hundreds, if not thousands, of compute threads and increase application resilience; since, one or two downed nodes in a cluster of hundreds of nodes doesn’t have the same impact as one or two large virtualization hosts within a cluster of tens of nodes.

One new offering in the ARM server market is ZT System’s R1801e 1U rackmount chassis that houses eight nodes (or “System on Modules” in ZT-speak), one 80GB SSD per node, consolidated network by way of two integrated 4+1 Gigabit Ethernet switches, and is pre-loaded with Ubuntu Server. Each node has an ARM processor with dual Cortex-A9 cores, 1GB of DDR3 memory, 1GB of flash and the typical USB and SATA connections found in common systems.

The nice thing about a configured chassis is that it has a power budget if 80W or less (which may be a bit optimistic). That’s less than the TDP of a common six-core Xeon 5600-series processor. Granted, each of the Cortex-A9 cores cannot match the performance of just one of the six cores in a Xeon 5600-series processor, but it provides a good amount of punch in a highly distributed compute environment.

Still not convinced of the performance of a cluster of low power compute nodes? Look at the specs for the IBM BlueGene/L supercomputer. Each node board contains sixteen compute cards, which each card contains a pair of PowerPC 440 processors running at 700MHz. Doesn’t seem like much initially, but scale that out to hundreds of node boards and you have a supercomputer in the making. Granted, comparing a portion of a BlueGene/L installation against a cluster serving up web content and applications isn’t apples-to-apples, but it provides a glimpse of what is possible when you aggregate a lot of low power nodes together.

So, there has to be a gotcha with any new shiny toy, right? Well, there is one big one: cost. Each 1U chassis with eight nodes and eight 80GB SSDs will cost you around $20,000. Some of that cost is due to the fact that it is currently a low volume product, so economies of scale isn’t there yet. One can theorize that swapping out each of the SSD with a traditional hard drive would help reduce the cost of the system, but now you have re-introduced a moving part back into the mix and increased power consumption by around 2-4W per node.

The only other potential limitation might be the 1GB of memory for each node, though that can be mitigated by running lightweight web servers (such as: Cherokee or lighttpd) or using a distributed memory model.

The next step in this particular market would be to integrate a Marvell Armada XP processor or higher-end Cortex processors on to each node and either stick with externally accessible drive bays or go with flash modules that directly attach to each node’s board.