Among web developers, anticipation is mounting for HTML 5, the overhaul of the web markup language currently under way at the Worldwide Web Consortium (W3C). For many, the revamping is long overdue. HTML hasn't had a proper upgrade in more than a decade. In fact, the last markup language to win W3C Recommendation status, the final stage of the web standards process, was XHTML 1.1 in 2001.

In the intervening years, web developers have grown increasingly restless. Many claim the HTML and XHTML standards have become outdated, and that their document-centric focus does not adequately address the needs of modern Web applications.

HTML 5 aims to change all that. When it is finalized, the new standard will include tags and APIs for improved interactivity, multimedia, and localization. As experimental support for HTML 5 features has crept into the current crop of Web browsers, some developers have even begun voicing hope that this new, modernized HTML will free them from reliance on proprietary plug-ins such as Flash, QuickTime, and Silverlight.

But although some prominent Web publishers -- including Apple, Google, the Mozilla Foundation, Vimeo, and YouTube -- have already begun tinkering with the new standard, W3C insiders say the road ahead for HTML 5 remains a rocky one. Some parts of the specification are controversial, while others have yet to be finalized. It may be years before a completed standard emerges and even longer before the bulk of the Web-surfing public moves to HTML 5-compatible browsers. In the meantime, developers face a difficult challenge: how to build rich Web applications with today's technologies while paving the way for a smooth transition to HTML 5 tomorrow.

Modernising HTML for the rich web

Rich applications and HTML have not always been a natural fit. The father of the web, Tim Berners-Lee, envisioned HTML as "a simple markup language used to create hypertext documents that are platform independent." With the advent of XHTML, the pure XML formulation of the language, the W3C maintained this focus on web pages as documents, with the proposed XHTML standards emphasising such issues as document structure, compatibility with XML tools, and Berners-Lee's vision of the Semantic Web.

This frustrated many developers who saw greater potential in the web as an application platform. In 2004, representatives of Apple, the Mozilla Foundation, and Opera Software founded the Web Hypertext Application Technology Working Group (WHATWG), an independent web standards consortium. Working outside the W3C, WHATWG began a parallel effort to revamp HTML for a more application-centric view of the web.

In 2007, with its XHTML 2 work mired in seemingly endless debate, the W3C voted to adopt WHATWG's work as the starting point for a new HTML 5 standard. By this time, even Berners-Lee had come around to the notion of an application-centric web. "Some things are clearer with hindsight of several years," he wrote in 2006. "It is necessary to evolve HTML incrementally. The attempt to get the world to switch to XML... all at once didn't work."

That's not to say the concept of a pure-XML web markup language is dead. Although HTML has retaken the lead role in the standards effort, an XML formulation of HTML 5, to be known as XHTML 5, is being developed at the same time. The difference is that while XHTML 5 will be available for those who have already made the switch, developers will no longer be required to observe the rigorous syntax of XHTML to take advantage of web markup's latest features.

HTML 5: Markup gets a makeover

Be that as it may, HTML 5 has inherited many additions originally proposed for XHTML 2, including a number of features designed to improve document structure. For example, new HTML tags such as header, footer, dialog, aside, and figure allow content authors to specify common document elements in a consistent way. Previously, developers had to mark such elements using div tags with custom class attributes, an arbitrary method that made HTML documents difficult to parse.

HTML 5 also continues the effort to separate web content from presentation. Developers might be surprised to see the b and i elements available in the new standard, for example, but these elements are now used to offset portions of text in generic ways, without implying any specific typographic treatment. Where the i element once implied italic type, for example, in HTML 5 it merely means "a span of text in an alternate voice or mood." Similarly, the b element does not imply specifically boldfaced text, but text that is stylistically offset without having any additional importance.

By comparison, the u tag, which referred specifically to underlined text, has been dropped from HTML 5, along with other presentation specific elements, including font, center and strike. Such stylistic attributes are now considered the exclusive domain of CSS.

The new standard introduces additional data types for form input elements, including dates, URLs and email addresses. Still other elements improve support for non-Latin character sets, including tags for specifying the "ruby text" that appears in some Asian languages. HTML 5 also introduces the concept of microdata, a method of annotating HTML content with machine readable tags, making it easier to process for the Semantic Web. Together, these structural enhancements allow content authors to build cleaner, more manageable web pages that play nicely with search engines, screen readers and other automated content parsers.

Enabling a richer, standards-based web

But the most eagerly anticipated additions to HTML 5 are the new elements and APIs that enable content authors to create rich media using nothing more than standards-based HTML. Modern web pages increasingly incorporate scalable graphics, animation, and multimedia, but so far these capabilities have required proprietary plug-ins such as Flash, RealMedia and QuickTime. Such plug-ins not only introduce new security risks, but they also narrow the audiences of the resulting pages.

One way HTML 5 solves this problem is by aligning itself more closely with related markup languages. Content authors can embed markup written in MathML (for rendering equations) and SVG (for rendering scalable vector graphics) directly into their HTML 5 markup. This increased flexibility makes cross-platform HTML more competitive with file formats such as Flash and Silverlight, which were designed with both text and graphics in mind.

But web developers are clamouring loudest for HTML 5's new audio and video tags, which aim to finally make it easy to embed multimedia content into web pages. These tags are defined in the HTML 5 standard as being codec neutral, meaning it's up to individual browser vendors to support the codecs needed to play any given content item. Still, the video tag in particular is expected to be a godsend, particularly for online video providers who want their content to be available on Apple's iPhone and forthcoming iPad, neither of which supports Flash.

Taking interactive web graphics one step further is the canvas tag, which can be used to define areas of the browser window as dynamic bitmaps. Web developers can use JavaScript to manipulate the content of canvas elements, rendering graphics in real time in response to user actions. In theory, this technique could allow developers to create fully interactive games using nothing more than JavaScript and HTML.

In addition to these onscreen technologies, HTML 5 also introduces the concept of browser-based application caches, which allow web applications to store information on the client device. Like the Google Gears plug-in, these caches can both speed up application performance and allow users to continue to use Web applications even when they do not have access to the Internet. In fact, Google is already planning to phase out support for Gears in favor of the HTML 5 technology.

Browser plug-ins: Not dead yet

But for all of HTML 5's new features, users shouldn't expect plug-ins to disappear overnight. The web has a long history of many competing technologies and media formats, and the inertia of that legacy will be difficult to overcome. It may yet be many years before a pure-HTML 5 browser will be able to match the capabilities of today's patchwork clients.

For example, while Vimeo and YouTube are already experimenting with the HTML 5 video tag, deploying HTML 5 multimedia will not be as easy as it sounds. The W3C's decision not to specify media codecs in the HTML 5 standard means developers cannot guarantee that any one media format will be playable on every possible client device. Apple, Google, and Microsoft are pushing for H.264 video, for example, but open source browsers such as Firefox and Konqueror are unable (or ideologically unwilling) to licence the appropriate patents to support that format. Unless this boondoggle can be resolved, web content authors who need to reach the widest possible audience may be forced to continue to rely on Flash.

Not every legacy application will be rewritten for HTML 5, either. For example, even if Google downplays its own Gears technology in favor of a standards-based approach to local application storage, the Gears API and the HTML 5 application cache API are not identical. Google itself admits that "there is not yet a simple, comprehensive way to take your Gears-enabled application and move it (and your entire userbase) over to a standards-based approach." Until there is one, even users of fully HTML-compliant browsers may be forced to install Gears to support some legacy applications.

In the end, browser market share may be the most significant hurdle for developers interested in making the most of HTML 5. Internet Explorer 6, for all its rendering quirks and inept handling of web standards, is seemingly the browser that cannot die. Older versions of Firefox, IE, Opera, Safari, and others all have large user bases, and none support HTML 5. Until these legacy browsers are replaced with modern updates, web developers may be stuck maintaining two versions of their sites: a rich version for HTML 5-enabled users, and a version for legacy browsers that falls back on outdated rendering tricks.

In HTML 5's favor, Apple's iPhone and iPad will not support Flash, but are expected to gain support for HTML 5 features as the standard matures. Similarly, Google's Chrome browser leads the pack in HTML 5 support, and devices based on the company's forthcoming Chrome OS are expected to follow suit. Large web publishers, however, have traditionally been conservative about standards support. Even given a large HTML 5 installed base, it may be years before the Fortune 500 is willing to risk the upgrade.

How to try HTML 5 today

Some voices among the web development community also urge caution. Although Microsoft plans support for HTML 5 in Internet Explorer 9, for example, the software giant questions the wisdom of claiming support at this early stage. "Saying you are standards-based but then saying you are the most HTML 5-compliant browser does not make sense, because the standard is not [complete] yet," Microsoft's Steven Sinofsky remarked in a recent interview.

Indeed, no organization is more guarded in its estimates of HTML 5 adoption than the W3C itself. The HTML 5 working group does not expect the standard to reach Candidate Recommendation status, the feature complete phase of the W3C standards process, before 2011. Even then, the process of ratifying the standard as a W3C Recommendation is expected to continue until somewhere around 2022. If you're doing the math, that's 21 years from XHTML 1.1 to HTML 5.

By any count, HTML 5 is likely to remain cutting-edge technology for the next five to 10 years. Early adopters who would like to see it in action today can do so, however, albeit in a limited way. A number of pilot projects and demonstration sites that showcase the various capabilities of the new standard are available online; the key is choosing the right browser. Support for HTML 5 features in Firefox is spotty. Browsers based on the WebKit rendering engine, including Chrome and Safari, work best. Ironically, that means Internet Explorer is also an option, but only with the Chrome Frame plug-in installed.

Web developers, likewise, are free to experiment. Whole sites can be built with code that conforms to the current draft of the HTML 5 specification, although results with current browsers will be spotty. One of the best online resources for would-be HTML 5 developers is Mark Pilgrim's excellent Dive into HTML 5, which includes, among other things, a detailed guide to navigating the complex world of the HTML 5 video element and the various codecs supported by current browsers.

So much work remains to be done on the HTML 5 standard, however, that some organisations are liable to dismiss it as yet another overhyped, up-and-coming technology. That would be a mistake. Standards bodies by their very nature move slowly, but work on HTML 5 is being driven by large, motivated vendors, including Adobe, Apple, Google, Microsoft, the Mozilla Foundation, Opera Software, and others. These companies recognise the need for an upgrade to the HTML standard, and their work is helping to realise its potential. The resulting opportunities for web developers are too compelling to ignore.