As we enter the second quarter of 2007, for many enterprises it's time to cash in on VMware's ESX upgrade promotion and begin moving their virtual infrastructures from ESX Server 2.X to VMware Infrastructure 3 (VI3). VI3 offers enhancements over its predecessor, each of which provide compelling reasons to consider an upgrade.

Enhancements

The new VI3 architecture offers more firepower for enterprise class virtual machines. Instead of being limited to two virtual processors and 3.6GB of memory per VM, users can now configure guests with up to four virtual processors and 16GB of memory.

A number of new features in VI3 are also proving to be key factors in pushing users down the VI3 upgrade path. VMware Distributed Resource Scheduler (DRS) provides a mechanism to automatically migrate virtual machines to balance utilisation and guarantee resource availability within the overall virtual infrastructure. VMware High Availability (HA) provides automated protection of virtual machines during host failure and analyses resource utilisation across the infrastructure to determine the most suitable host on which to restart affected VMs.

The final major enhancement to VI3 is VMware Consolidated Backup (VCB). This offering allows users to offload virtual machine backups from the ESX Hosts and from within each virtual instance. Instead, backups are performed using the resources of a Windows Backup Proxy. This process allows virtual machine backups to occur over a SAN which drastically decreases the amount of time necessary to backup a virtual infrastructure.

Beyond these highlighted features, VI3 also provides enhanced performance over previous versions of VMware ESX Server.

Migration practicalities

But the need to upgrade doesn't come into question as often as the actual process of upgrading. The key to a successful virtual infrastructure upgrade is in the proper management of each phase of the migration process; pre-migration, and the actual migration itself.

The most critical aspect of a virtual infrastructure upgrade falls within the pre-migration planning phase. This phase forces customers to ensure they have a solid understanding of their virtual infrastructures. The first thing that every organisation should do is create an inventory of their environment. Many people will be quite surprised by how much larger their virtual infrastructures are compared to their original designs.

This inventory should contain the following items:

  • Names and count of all ESX Hosts
  • Names and count of all virtual machines
  • Names, count, and sized of each VMFS volume
  • Names and count of each virtual switch

A typical virtual infrastructure will have many interdependencies between configured virtual machines. Mapping out these virtual machines and linking them to their dependencies is a critical step as it provides a way to perform an impact analysis before beginning the upgrade process. Special care should be taken when dealing with virtual machines on which a high number of other VMs have a dependency. Any problem during the migration of a critical VM can have a serious knock-on effect on other dependent VMs.

In-place upgrades

At this point in the process it is important to mention that there are actually two different paths that can be followed for upgrading a virtual infrastructure; in-place upgrades and migration upgrades. In-place upgrades involve selecting a host machine and all the VMs that reside on it for a complete migration at the same time. While this can appear to be a simpler and less cumbersome method, the risks associated with moving a host and a large number of VMs at once are far greater. Experience up to this point has proven that in-place upgrades should be performed only when there is absolutely no choice. The potential impact of a failed in-place upgrade of an ESX Host or VMFS volume with configured virtual machines can be devastating to a virtual infrastructure. If an in-place upgrade is the only option, performing a backup of all virtual machines running on a host can protect against possible data (and job) loss.

The migration upgrade path provides a greater level of flexibility and decreases the downtime necessary to move individual or smaller groups of virtual machines from ESX 2.X to VI3. Migration upgrades are definitely the preferred approach to migrating to VI3, but this path does require that at least one additional ESX server and one additional VMFS volume are available to configure with ESX 3 and VMFS 3, respectively.

Dependencies and support

The next step after determining the migration path is designing a schedule based on the interdependency map that was previously created. The most important aspect that should be considered when creating the schedule is to ensure that each application will be potentially affected only once. For example, if an application server is migrated one week, and the database server that feeds data to the application is moved the next, there is an unnecessary risk introduced into the migration process. It may not be possible to eliminate these occurrences, but every attempt should be made to minimise them by moving dependent machines at the same time.

The final step that should be performed before executing any migrations from ESX 2.X to VI3 is ensuring that support applications will properly function in the VI3 environment. These applications include, but are not limited to, backups, replication, monitoring, and provisioning. A major problem encountered by early adopters of VI3 was a lack of support tools for the management of upgraded infrastructures. It is important that organisations verify support with the vendor and download the newest versions of the support applications if necessary.

Migration options

Upon publication of the upgrade schedule the actual migration process can begin. The first step is preparing the VI3 infrastructure to support the migrated virtual machines. This preparation involves two items; introducing VirtualCentre 2.0 and building an initial ESX 3 host.

The introduction of a VirtualCentre 2.0 server is often a controversial issue for many organisations that can be handled in one of two ways. The most common way to introduce VirtualCentre 2.0 to an infrastructure is to upgrade the existing VirtualCentre server from version 1.X to 2.0. This method allows users to keep their historical performance data about the virtual machines running in their infrastructure. When a VirtualCentre server is upgraded it becomes the central management tool for both the existing 2.X environment and the new VI3 infrastructure. As with any other major step of the migration process it is important to get a good backup of the VirtualCentre database before proceeding with the non-reversible upgrade process.

A less common option for introducing VirtualCentre 2 into an environment is to build a new standalone system and database. This option will allow for the separation of management tasks between the old and new environments. The major downside to this process is that all historical data about each virtual machine is not available within the VirtualCentre 2.0 database and will be lost upon full migration.

With a VirtualCentre 2 server in place it is time to build the first ESX 3 host that will host migrated virtual machines. Users need to take extra precautions when building out ESX 3 hosts. Many defined best practices have been modified between ESX versions in regards to partitioning, console support applications and security configurations. Organisations should make use of the proper engineering resources as necessary to ensure ESX 3 is properly configured for their environment.

While defining the new ESX 3 standard, many organisations will find that this is a prime opportunity to modify larger virtual infrastructure configurations. Some organisations may determine that their infrastructure has grown to such a size that VLAN tagging on their virtual switches becomes a much better way of managing their increased virtual machine estate.

Other organisations may find that their LUN sizing should be modified to better fit future operating system and application configurations. Even a task such as modifying the naming standards for specific components can be easily done during the migration process. This portion of the VI3 migration process provides a clean slate for these types of modifications to be implemented with no impact to the running infrastructure.

Automation

Once a VirtualCentre 2 server and at least one ESX 3 host are in place the actual process of migrating servers begins. Like every other step up to this point there are several ways of approaching this. The actual process of migrating a virtual machine has the largest potential impact on a virtual infrastructure. There have been many hard lessons learned due to poorly performed migration steps, so choosing the proper method of migration is critical.

The key to a successful migration of the virtual machine environment is to use as much automation as possible. Look for solutions that provide proper scheduling to control exactly when a virtual machine will cut over into the new VI3 environment. It is important to note that there is no such thing as a "Zero Downtime" migration. At some point the migrated virtual machine will need to be shut down to upgrade the virtual hardware, which allows the system to switch over to VI3. Additionally, the VMware Tools installer will need to be run after upgrading the virtual hardware, which requires a second reboot of the system. It is best to schedule the downtime of a virtual machine to complete the entire process within a single change window.

Minimising downtime

There are several solutions available to help automate the upgrade process and minimise the amount of downtime required to successfully migrate a virtual machine. While researching solutions for virtual infrastructure migrations there are several key items to consider:

  • Ensure it is still possible to easily fail back to the ESX 2 environment if a migration fails. The migration process should not modify the source ESX 2 virtual machine in such a way that it is rendered irrecoverable.

  • A virtual machine migration solution should have a definitive cut-over window. It is difficult to migrate a virtual machine properly without having control over it when it switches to the VI3 environment.

  • It should be possible to modify any configuration of the virtual machine as part of the migration. Changing naming standards of virtual switches and other configurations is a natural part of the migration process.

  • Migrations should be able to occur to and from any type of storage subsystem. Many customers may want an opportunity to upgrade from local to SAN storage or from SAN storage to a lower cost iSCSI array.

Many organisations are faced with significant challenges during the planning and execution of a virtual infrastructure upgrade. Creating a solid plan before beginning the migration process will help avoid many of the common pitfalls. Approaching the upgrade with the proper procedures will provide the easiest upgrade path with the lowest amount of risk to the infrastructure.

Finally, organisations that arm themselves with the proper tools will have far greater control over the process and will be likely to experience far less disruption to their normal operations. While upgrading a virtual infrastructure can be challenging, it does not mean it needs to be a risky and troublesome process. By planning the upgrade process properly and making use of available tools, customers can benefit from the numerous enhancements offered by VI3 without suffering the headaches and protracted bedding-down periods so often associated with migration projects.

Scott Herold is director of research & development at Vizioncore.