Three years ago, Sesame Workshop, the non-profit organisation behind the "Sesame Street" television show, was looking at a $3 million data centre expansion to keep up with its web, multimedia and data storage needs.

Instead of expanding, however, the organisation shrank its data centre by consolidating 100 physical servers to 45 Novell SUSE Linux Enterprise Servers, then virtualising 25 data centre servers into five physical machines.

"That data centre has extra space now. And we're paying less for heating and cooling," says Noah Broadwater, vice president of IS at Sesame Workshop, in New York.

Like other forward-thinking IT managers, Broadwater espouses virtualisation's obvious blessings: A company can spin up and move around virtual-machine images to meet load and use requirements at the speed of business while drastically reducing hardware, storage and cooling costs.

Unfortunately, this very convenience is a curse in terms of manageability, something Broadwater and others say can spiral out of control quickly if the rollout isn't planned properly and the virtual-machine life cycle taken into account. "I can't see consolidating 25 physical servers down to five and then needing three management servers to run it all. We've stayed away from all that," he says.

Most IT organisations have a hard time nailing down their physical inventory. They lose track of things over the years and through mergers, agrees Jim Houghton, who led Wachovia's Corporate & Investment Bank IT Utility group until a year ago and now is CTO of Adaptivity, an infrastructure consulting firm. For example, when Wachovia's IT department initially deployed Tideway Systems' Foundation application and discovery tool, "we found over 50 servers that should have been retired," he says.

Managing dynamic application-instances from birth to grave takes more than those procedures required to manage physical servers and desktops, Houghton says. For example, what of the virtual applications (or the composite of multiple applications) running on those servers?

IT executives who have virtualised their infrastructures say they've been compelled to dedicate one or more staffers for management tasks and to cross-train their staff at the systems and network operations levels. Plus, they say, they've felt pressured to choose between best-of-breed point solutions or large, enterprise management frameworks.

Management style

Users are reacting to this early and fragmented tools market by managing their infrastructures in layers - as shown in the results of a survey of Network World readers. Of the 335 respondents who indicated they have some type of life-cycle management in place for their virtual environments, 61% said they use two to five tools; only 18% rely on a single virtual-machine management tool.

In a larger base of 522 respondents, 48% use the native management capabilities of the Citrix Systems XenSource, Microsoft Hyper-V and VMware ESX hypervisors, and 10% layer third-party products into the mix. (Thirty-one percent do no monitoring at all.) When it comes to management automation, nearly two-thirds of 358 respondents cite tie-ins to traditional systems-management platforms from such companies as BMC Software, CA and IBM Tivoli.

"Life-cycle management gets complex when you start getting into layers and feeding into big system-management interfaces," Sesame Workshop's Broadwater says.

To keep management under control, Sesame Workshop uses the Xen virtualisation hypervisor native to the SUSE Linux machines on which the company has standardised. Systems managers also use Novell ZENworks Orchestrator and Asset Management tools to keep tabs on the virtual machines.

While planning their virtual-machine infrastructures, organisations need to look at management as a core component, experts say. So, along with inventorying physical servers and applications and creating gold-build images, they should consider management options as they pertain to usage requirements, says Leslie Muller, formerly a senior technologist for virtualisation deployments at Credit Suisse and now CTO of DynamicOps, a virtual-machine orchestration company.

"You need to consider the type of worker. Is he or she a knowledge worker who requires a specialty-build on-demand? Or is he or she a call center worker who needs the same environment day in and day out? In the latter case, a standard virtual-machine build would be in order," Muller explains.

Vignette, a software publishing company, follows the "on-demand" philosophy for 200 developers and other employees around the world. The company has resources running on 200 virtual-machine images in Austin, as well as 100 in Australia, 22 in India and a handful in the United Kingdom.

"Our development environment is extremely dynamic. When our guy in India goes to bed at night and our guy in Texas wants to use that virtual-machine slot, the system needs to enable this," says David Graham, CIO at Vignette. "Management needed to fold into the behavior of users requiring this level of access."

A beta version of Hyper9's Virtual Infrastructure Search and Analytics tool, a Google-like search engine that locates and takes reads on the state of virtual machines it finds, shows what's happening with the organisation's virtual machines as they're being used, shelved and reused. In addition, VMLogix laboratory-automation software helps Vignette script and monitor the use of the virtual machines.

"Locating physical machines, the virtual machines within them; knowing who set them up, what they're used for, who's changing them; knowing if they need to be erased, where to store them - these are all areas the developers can manage themselves," Graham says. "Our developers need that level of control."

Nixon Peabody, a global law firm with 1,700 users, has a more centralised approach to virtual-machine life-cycle management. Operations team members use VMware's inherent management features to track and control the specific production, job, SQL and development servers for which they're responsible.

Plus, the firm has dedicated a VMware Certified Professional (VCP) to the overall care and feeding of 140 virtual servers running on 13 physical machines in its New York data center, says Peter Allen, director of IT operations at Nixon Peabody.

The VCP uses Fortisphere Virtual Essentials management and policy enforcement for overall monitoring and control. The Fortisphere tool discovers all the virtual machines running in an environment; gives a history of those virtual machines; and allows for change, configuration, performance and inventory management without requiring an agent on each virtual machine, Allen says. Fortisphere also supports security at the build and configuration management stages, he adds.

"We needed this tool for the whole change-management process. We want to be alerted when something's changed within the environment," Allen notes. "We've done this without too much layering of tools at this point. And because Fortisphere charges per physical server, we thought that was a reasonable model for our environment."

What Fortisphere doesn't have is the ability to find idle or powered-off machines, but it is working on such capabilities, Allen adds.

Of course, virtual-machine platform tools can be told manually to end the life cycle of a given virtual machine, as long as the system manager is informed and paying attention. What Allen refers to is the automated process of integrating end-of-life rules across multiple virtual-machine brands and types.

For this level of management, Informatica, a data-integration software maker in California, is considering FastScale Technology's FastScale Composer Suite. With this tool, IT should be able to manage its physical and virtual environments, including the life cycle of deployed applications, their duration and the reprovisioning of server space once a job is done, says Tony Young, CIO at Informatica, which uses 350 virtual machines.

"I need to let my developers log into a portal and say, 'I need this application for this amount of time,' and when that date hits, they're pinged that the machine is about to die, and 'do you want to renew?'" Young explains. "With FastScale, I can catalog and reprovision all of my virtual machines across my Linux and Windows worlds."

Keep it basic

As this sampling of use cases shows, you won't find a universal approach to managing virtual environments. In fact, preparing for heterogeneous virtualisation is the hardest stage of the virtualisation movement, says research firm Gartner in a May research paper on virtualisation planning.

"My analogy is to stop thinking of a virtual machine as anything other than a server," Sesame Workshop's Broadwater surmises. "Start slowly, know your environment, plan for allocation and benchmarking and utilisation. And set a lifespan, just as you would for a physical server."

Radcliff, a freelance writer, can be reached at [email protected].