It's an odd juxtaposition. The Dell PowerEdge R715 sits in a rack just below a recently decommissioned 2U, two socket server that cost about as much when it was new five years ago. The difference? The older server has a total of two CPU cores, one per processor. The R715 has 32 cores, running at the same clock speed. If that's not progress, I don't know what is.
The R715 has been around since February, so strictly speaking, it's not a new server version. What's new: the R715's twin 16 core AMD Opteron "Interlagos" CPUs and support for 1,600MHz DDR3 RAM. Best of all, existing R715 systems can be upgraded to support the new chips with nothing more than a BIOS update. Otherwise, Interlagos is a drop-in replacement for Magny-Cours.
Like Magny-Cours before it, Interlagos promises to bring unbeatable price/performance to heavily multi-threaded workloads such as virtualization. It costs considerably less than its closest Intel counterparts and offers twice the cores of the leading Intel chip. However, Interlagos is based on a new core architecture that is not yet supported by a number of popular operating systems (more on that below).
AMD Opteron 6200: Big guns, new design
The AMD Opteron 6200-series Interlagos CPUs have 16 cores and 16 threads, each of which has a dedicated core. However, as the first Opteron based on AMD's new modular architecture, the Interlagos chip is built from eight Bulldozer modules. Each module, in turn, has two independent processors, but the FPU, fetch, decode and execute units are shared. It's not quite the same as having 16 completely independent cores.
That said, Interlagos also supports Advanced Vector Extensions (AVX), which essentially double the width of the FPU, from 128 to 256 bits. The AVX extensions should produce greater floating point performance, but only if the code explicitly supports them. Whether these changes result in a performance boost or a bottleneck will be highly dependent on the workload, as heavily threaded processes lacking AVX support that hit the FPU frequently may see some slowdown.
Clearly, the Bulldozer architecture is significantly different than AMD's previous major CPU iterations, and AMD appears to be positioning new architectures for the future to reduce the dependency on the FPU. AMD's Fusion system, which essentially moves FPU duties to a GPU, would be one example. It may be that whatever bottlenecks are present now will be mitigated later once various libraries and applications have been updated to take advantage of these new hardware designs. One definite bonus with Interlagos (and Bulldozer in general) is the addition of the AES-NI instructions that dramatically reduce computation times for AES encryption.
Unlike previous Opterons, the Interlagos chips offer AMD's Turbo Core technology, which allows the CPU to boost the clock rate by 300MHz to 500MHz across all cores or up to 1GHz across half the cores. Thus heavily threaded loads will see the smaller but broader boost, while less-threaded workloads will see the higher clock speed but on fewer cores.
One casualty of the architecture changes is that several operating systems simply will not run on Interlagos by default. These include Windows Server 2003 prior to R2 SP2, RHEL 4.x, RHEL 5.0 through 5.6 and RHEL 6.0, Novell SLES 10 through SP3 and SLES 11 and any Linux variant running a 2.6.31 kernel or earlier. The latter group includes VMware ESX 3.5 and VMware ESX 4.0 through 4.1u1.
A number of other operating systems will run but won't take advantage of all the new capabilities. RHEL 6.1 and SLES 11 SP1 will take advantage of the AES-NI instructions, but not the AVX extensions or Turbo Core. Windows Server 2003 R2 SP2, Windows Server 2008 SP2 and VMware ESX 4.1u2 will run but won't take advantage of the new instructions or features.
Then there's the list of operating systems that will handle both the new features and the AES-NI instructions, such as VMware vSphere 5.0, Windows Server 2008 R2 SP1 and RHEL 6.2. These are only a few examples of operating systems with full support, and some of these OS versions are still in beta. It's important to know whether or not your core operating systems are supported on this chip before you make any purchasing decisions.
It's also important to note that a compatible hypervisor, such as VMware vSphere 5.0, will allow virtualised instances of incompatible operating systems, such as RHEL 5.5, to run on an Interlagos box.
Dell PowerEdge R715: Interlagos edition
The Dell R715 is a 2U rack-mount server with redundant power supplies, four gigabit Ethernet ports and a DRAC6 Express or Enterprise embedded management controller.
With six 2.5 inch hot-swap drives on the front, it can support up to 6TB of raw storage, backed by Dell's PERC H200 or PERC H700 internal RAID controllers. There's a limit of 256GB of RAM and for the moment the 1,600MHz DDR3 RAM is limited to 4GB and 8GB DIMMs, 16GB DIMMs are expected to be available soon.
The inclusion of redundant SD card slots permits installation of an embedded hypervisor that allows the R715 to function as a virtualisation host server with no need for physical disks. All in all, it's a solid server.
I had only a few days to work with the refreshed Dell R715 before publication. The test unit was equipped with two AMD Opteron 6274 chips running 16 cores at 2.20GHz per core, 16MB of L2 cache, 12MB of L3 cache and 64GB of 1,600MHz DDR3 RAM.
I verified that the slightly older RHEL and SLES releases blow up during the installer boot, and that RHEL 6.1 and the RHEL 6.2 beta install and run as you would expect. Windows Server 2008 R2 SP1 and VMware vSphere 5.0 also run fine. Unfortunately, as most of my historical CPU benchmarks run on Linux, and the new instructions are not supported on shipping versions of RHEL, those results will have to wait for a follow-up article.
Dell has provided some SPECcpu2006 and SPECjbb2005 benchmark results that show the Interlagos chip beating its Magny-Cours predecessor handily using code optimised for Interlagos. The SPECjbb2005 score was nearly 40 percent faster.
As with any significant change in architecture, it will take time for the Interlagos chips to enjoy broad OS support and show what loads they're capable of handling. Those in the HPC space are jumping on the Interlagos bandwagon, as evidenced by Oak Ridge National Laboratory pumping up its Jaguar system to a Cray XK6 running AMD Interlagos chips with some 25,000 processors.
The Dell PowerEdge R715 is a very sleek and streamlined server that now offers a fantastical 32 cores. It will find a home in HPC, database and certainly virtualisation farms. The ability to upgrade an existing R715 to the AMD 6200 series Interlagos chips is a definite bonus for those who've already deployed these servers, and it portends well for those signing purchase orders today.
The overall success of the R715 will depend on the success of the AMD 6200-series chips, and though the future looks promising, only time will be able to tell that story.