It's hard to believe that it's been almost a decade since I-Way [pdf], the first large-scale grid experiment. For those of us who were there at the genesis of the technology, it's been exciting to see more frequent grid-focused articles in IT trade publications as pilot projects are rolled out on a commercial scale.
From implementations by major US financial institutions such as Charles Schwab and J.P. Morgan, to equally intriguing grid efforts by companies such as Burlington Coat Factory and GlobeXplorer, we're at the onset of the enterprise adoption curve. These are indeed interesting times.
But early commercial grid discussions have been distracted by the onslaught of marketing hype around the category. Part of the difficulty is that the term grid is being adopted to label a whole range of things to manage the familiar bugaboos of cost and complexity, such as virtualisation, clustering and better management of IT resources. Vendors are finding it convenient to use grid as a term to describe the common themes at play.
I don't want to belabour the real definition of grid; you can find my definition in "What Is the Grid?" [pdf]. To me, the more productive conversation is the role of open standards and open-source in the early evolution of enterprise grid.
Yes, there already are and will continue to be some interesting proprietary products built on grid technology, but we shouldn't lose sight of the importance of standardisation and creating a foundation that end users can build upon to realise grid and virtualisation on an enterprise scale. It's not clear that you're going to get there by buying a proprietary solution, even if it's dressed up in a nice grid marketing package.
Grid is, on many levels, a glorified interoperability discussion. It's about making heterogeneous resources play nicely together, synchronised to meet business needs. That notion has been highlighted in vendor marketing material to the point of becoming trite, but the real promised land of grid isn't just interoperability but integration among resources -- and across geographic barriers -- within an organisation. Ultimately, it could enable integration between organisations, as is already common in the sciences. But we need to walk before we can run.
The key to this progression is to start looking at your IT infrastructure not as a collection of proprietary boxes, but rather, as a set of services. Then, you can start doing dynamic provisioning of those services and thinking in terms of the autonomic directions that IBM has been vocal about. Given some time, grid can become the underlying infrastructure that enables service-oriented architectures, virtualisation, utility computing and a number of other common visions that you're hearing so much about. In order for these things to work, you have to have good distributed management functionality -- and that's what grid can provide.
Think about what grid means. It means connecting a bunch of distributed resources. If they're all point proprietary solutions, we'll never get over the interoperability hump. So, common interfaces are key.
But we don't have to wait for standards to be defined and vendors to implement them. There's a reason why grid is an open-source movement, and it's analogous to the explosive growth of Linux in the enterprise in recent years.
Companies like Google, which has based its entire massive infrastructure on Linux, found open-source highly appealing from a commercial perspective. It's a question of speed and flexibility. Systems can be deployed quickly, and new equipment can be integrated easily. Standards, as they are defined, are implemented rapidly. And above all, you can't overstate the importance of being able to peer into the base operations layer to modify and improve source code.
So in my opinion, enterprises will reach true interoperability much faster if they use open-source as the base-level grid infrastructure. In fact, many vendors' proprietary grid offerings already build on the foundation provided by the open-source Globus Toolkit.
Vendors of proprietary hardware and software need not fear for their businesses, as IT buyers won't buy any fewer of their products. In fact, grid pioneer Wolfgang Gentzsch at MCNC (formerly with Sun Microsystems) witnessed first-hand how allowing customers to cluster Sun hardware spurred companies to do new things with those resources, unleashing a new level of creativity that led to customers buying more hardware to feed the fire.
Grid's legacy is in science and academia, as well as its track record, is strongest in high-performance computing contexts such as massive number crunching, data analysis and simulation. Such applications can be important in the enterprise too, but a question at the centre of the commercial grid adoption story is how well and how soon grid can move into broader commercial application, and at what point organisations will be able to use grid in mission-critical, transactional situations.
This is an evolution with enormous implications for the holistic enterprise and the end-to-end network. And as described in a recent article here on Techworld, it's going to require broad IT knowledge to harness grid in the enterprise.
It's also going to require broad technical and cultural discussions, so I want to hear from you in the months ahead. What is your organisation experiencing in its pilot grid implementations?
Ian Foster is a co-founder of the Globus Alliance and an advocate of open-source grid computing. He can be reached here.