In our test of Microsoft's recently released 64-bit edition of Windows Server 2003, we found that when you employ optional, kernel-mode processing features, the operating system flies. When you don't, it runs a bit slower than other 64-bit server operating systems we've tested recently.
These Windows 2003 Server x64 kernel options let certain processes run at the kernel code level - in our test case SSL certificate processing, caching and session handling. When you combine these options with mandated 64-bit hardware drivers and the vast amount of memory that a 64-bit processor can address, you can get some of the best performance we've seen on Intel/AMD architectures.
When we used kernel SSL processing, the number of sustained users climbed by 90 per cent over 32-bit Windows Server 2003 processing. When compared with other 64-bit operating systems (Red Hat Enterprise Linux 4.0 [RHEL 4.0] Advance Server and Solaris 10), Windows Server 2003 x64 has a 15 per cent to 20 per cent performance advantage.
Without the kernel processing options, Windows Server 2003 x64 performed slightly under par with competitive 64-bit operating systems in our testing.
The downside to these performance gains is incompatibility issues in terms of the hardware Windows Server 2003 x64 can run on and some of the applications it can support.
The two generic AMD64 white-box systems we tested were incompatible with Windows Server 2003 x64. One wouldn't start the kernel or boot through a kernel load. The other had constant crashes after installation that seemed to be related to motherboard memory timing and additional SCSI hardware driver issues.
Two systems provided by Microsoft OEM partners -- Polywell and HP -- had no operational issues. Our primary test server was HP's four-way Opteron DL585 server. HP was the only hardware vendor with a full array of hardware drivers posted at Microsoft's Web site when the 64-bit operating system was released in April. Buyers are captive to OEM hardware providers for now. This obviously limits hardware choice: something we didn't experience with the 64-bit editions of Solaris 10, SuSE SLES 9 or RHEL 4.0.
Old DOS and early 16-bit executables (games, WordPerfect 5.1, and Lotus 123 Version 4) didn't work at all or worked initially but then halted abruptly. Microsoft employs a 32-bit emulator called WOW64 that is automatically invoked to run 32-bit applications. We typically saw equal or slightly better performance of these 32-bit applications on Windows Server 2003 x64 vs. 32-bit Windows Server 2003.
Interpreted code, such as an old Visual Basic application we'd written long ago, worked very well on this 64-bit engine. And we could find no difference in execution time of a Perl script running on Microsoft's Internet Information Server Web service in the 32- or 64-bit Windows environments.
We developed an SSL Transaction script using Spirent Communications' Web Avalanche to gauge the number of sustained SSL transactions over a 10-minute build cycle (see How we did it, below).
The particular test ramps up the number of discrete user sessions, and then sustains sessions until the number of sessions dropped reaches one per cent. Generating SSL sessions is CPU-intensive, and managing multiple numbers of sessions presents a good gauge of how many balls the server can keep in the air before it drops one.
Using SSL to stress test Windows
We tested this script against Windows Server 2003 (both 32- and 64-bit versions) and compared these numbers against the 64-bit 2.6.7 kernel in RHEL 4.0 and Solaris 10 64-bit Edition. Both systems were running Apache 2.0.3 Web service with OpenSSL. We used default settings in all cases, except when we employed the kernel-mode SSL processing on Windows Server 2003 x64, as noted.
We took two sets of Windows Server 2003 x64 measurements: one reflecting the default kernel settings, and the other reflecting the aforementioned toggle that allows SSL to be processed by the kernel.
The difference between the results were startling, and proves the benefit of this simple setting. When we ran these tests on the four-way HP DL585 server, the operating system could sustain 288,471 sessions over a 10-minute period when the SSL sessions were handled at the kernel level. Microsoft states that the kernel lacks this setting by default, for backward compatibility reasons.
The Windows Server 2003 x64 native-kernel SSL session load was fast (207,202 sessions), but not as fast as RHEL 4.0 (251,024 sessions).
We also used two prior tests for comparison -- the number of maximum open TCP sessions, which measures how many can be sustained, and the number of TCP sessions per second each operating system could support, to gauge how fast the system can ramp them up.
In the maximum TCP transaction test, Windows Server 2003 x64 bested RHEL 4.0 but fell behind Solaris 10. In the TCP transaction per second measurements, Microsoft beat Sun, but fell behind Red Hat.
On the surface, little has changed between Windows Server 2003 32- and 64-bit versions, but performance numbers accentuate the musculature of the hardware and memory-addressing space beneath. Windows Server 2003 x64 is a strong performer, especially when nominally tweaked to take advantage of kernel-level options.
However, driver and hardware support is weak enough that Microsoft requires buyers to tap OEM-related vendors for sourcing Windows Server 2003 x64 products. When these issues are resolved, we'll give it a stronger recommendation -- because it's otherwise stable, fast and fully baked.
SSL and TCP stress-tests -- results
|OS tested||SSL sessions on|
4-way HP DL585
|SSL sessions on|
2-way Polywell 2200S/2
|Windows Server 2003 x64||86,545||207,202|
|Windows Server 2003 x64|
(with SSL kernel adjustments)
|Windows Server 2003 (32-bit)||n/a||151,902|
|OS tested||Max open TCP sessions||TCP sessions per sec. |
on HP DL585
|Windows Server 2003 x64 (native kernel)||166,217||2,505|
|Windows Server 2003 (32-bit)||88,116||1,276|
How we did it
We tested hardware-platform compatibility by testing the 64-bit Windows Server 2003 Enterprise x64 Edition on four platforms. We had no issues running the 64-bit code on our primary test platform, an HP DL585 server with four AMD Opteron 2.4-GHz CPUs and 12G bytes of dynamic RAM (DRAM). Nor did we have issues with the 64-bit code on our Polywell 2200S server with its dual AMD Opteron 2.8-GHz processors and 4G bytes of DRAM.
However, we had serious compatibility issues when we tried to run the code on an MSI motherboard-based white box system with one AMD Opteron 2.8-GHz processor and 1G byte of DRAM, and on an Asus K8N motherboard-based server with one AMD Opteron 2.0-GHz CPU and 1G byte of DRAM.
The official performance tests of Windows Server 2003 x64 were conducted on the HP DL585. We conducted tests in both its native mode and with its SSL-enhancements activated.
No other optimisations were used. The Windows server was both an Active Directory Domain Controller (and therefore DNS server), and a Certificate Authority. It was running Internet Information Server Version 6. A single, anonymous Web user was configured for all Web tests where applicable. No load balancing of any kind was used in any of the above tests.
We used the same HP DL585 server to test Solaris 10, Red Hat Advanced Server 4 and SuSE Linux Enterprise Server 9; all with default settings, and Apache 2.0.3 with OpenSSL (for SSL certificate services). DNS was deployed, but LDAP, Samba and Squid proxy were not used.
We tested performance with two Spirent Web Avalanche systems running in parallel.
The SSL tests were connected to the Web Avalanche systems by three Gigabit Ethernet connections; two from one Web Avalanche system, and one from the other.
The SSL test builds a set of user SSL sessions, via HTTPS reads from the server under test, until the number of concurrent users reaches a saturation point (generating 1 per cent errors), which we term Maximum Concurrent Sessions. This test severely exercises the CPUs in the system and requires management of the user session throughout the duration of the tests through "keep-alive" connection status "tickles" from the Web Avalanche boxes. The test is designed to exercise the Web services and operating system efficiencies until saturation. The duration of the test is 10 minutes, but saturation for all results arrived earlier.
We also performed TCP Open Connection and TCP Maximum Connection tests, and offer other prior test results for comparison.
The TCP Open Connection Test is a gauge of the number of TCP sessions that can be achieved per second, while the TCP Maximum Connection test measures the number of open TCP Connections that can be sustained until an error rate exceeds five per cent (in dropped connections).
Henderson is principal researcher for ExtremeLabs in Indianapolis. He can be reached at [email protected].
With the optional kernel tweaks, the system flies. While old 16-bit apps don't do well, newer, 32-bit code works fine. Real-world experience with 64-bit code in future will be the true test.