Beta testers for Microsoft's much vaunted XP upgrade are avoiding testing it because of its unpredictable effects across a whole range of applications.

The software giant sent out the latest trial version of its Service Pack 2 last week, but eight out of 12 users with at least partial XP deployments contacted by Computerworld last week said they had yet to begin evaluating it.

The security-focused service pack is due this summer and has been heavily promoted by Microsoft, but unlike traditional Windows service packs, SP2 doesn't simply roll up bug fixes that have come out since the prior release. It also includes new security features and enhancements designed to combat the spate of viruses and other threats that have hit the operating system. Other additions include a new Media Player and DirectX components.

"We're concerned about testing because we know it's not just patching that they're putting in there," said Bill Lewkowski, CIO at Metropolitan Health, a company with 1,300 users recently migrated to Windows XP with SP1. The company uses 400 applications against which to test OS upgrades.

"It just seems that Microsoft doesn't quite understand how difficult this is to do," said Lewkowski. "We can't do unplanned, unbudgeted service pack releases that are very similar to putting in a whole new version of an operating system," said Lewkowski. "I'm frustrated with Microsoft." Randy Truax, a technical services manager at Metropolitan Health said, "I almost prefer just fixes, because if they start putting in new functionality that developers have to test their code against, it adds a lot of complexity for us."

Several corporate users said they have either deployed SP1 or are deploying it as they bring in new PCs. But many still have mixed environments with a variety of older Windows versions.

Dave Chacon, technology services manager at golf club maker Ping, said 20 per cent of the company's 500 desktops run Windows XP SP1. Half have Windows NT4, 20 per cent Windows 98, and 10 per cent Windows 2000. Ping plans to deploy XP SP1 to all desktops by summer's end. But Chacon said it will wait at least six months to deploy SP2, since it anticipates that bugs will be discovered soon after its release. "We consider XP SP2 to be a major release because of the nature of the enhancements. XP SP2 is not simply an OS bug-fix release," he said. "Operating systems have reached a level of complexity that makes testing them for all possible configurations prior to general release difficult."

First American Title Insurance is excited about the new firewall and spam filter and plans to deploy SP2 as soon as its testing is completed, according to Scott Campbell, director of IT operations. But the company hasn't starting testing and is concerned about getting the configuration correct across more than 20,000 desktops. "We don't want to negatively impact the user communities' ability to access appropriate resources," Campbell said.

One company engaged in an enterprise-wide upgrade from Windows 2000 to XP is so confident about SP2 that it will soon have 1,100 PCs with release candidate 1 and 100 with the new release candidate 2. The CIO, who declined to be named due to the confidentiality requirements of Microsoft's early-adopter program, said the update files and executables for RC2 are 280MB, but what his company normally distributes to users via Microsoft's Systems Management Server is much smaller. "We have been comfortable deploying the SP updates via SMS automated updates, which don't require PC workstation visits to install," the CIO said.

Application breakage is a major concern for corporate users. But Keith Templin, a systems engineer at Cardinal Health, said his company has encountered no problems while beta-testing SP2. An IT manager at a large insurance company, who asked not to be named, also said that her company has had no problems, although it's anticipating some difficulty with NetMeeting and Windows Messenger.

Matt Pilla, a senior product manager at Microsoft, said users don't need to view SP2 as a "whole new version" of XP. He said it will be a significant amount of code and a large download, but it builds on the existing operating system. He said the technical preview program has drawn an "incredible response", and the size of the SP2 download will vary based on customer needs.

Since SP2 is a "smart download," only pieces that differ from what a customer already runs will be installed, he said. Users of SP1, for instance, won't need to download the fixes that it had bundled. Pilla was unable to provide an estimate of SP2's overall size.