Earlier this month, the US's TeraGrid project announced an additional $150 million in secured funding. The US National Science Foundation originally kicked off this project about three years ago, with the goal of creating a shared computing infrastructure that could be used by scientists nationwide.
Today, this massive grid connects eight large-scale computing centres across the US. Its 50TFLOPS (trillion floating-point operations per second) of computing power and two petabytes of floating storage are used for scientific discovery projects in domains such as chemistry, physics, and molecular and cell biology.
So what does TeraGrid mean to the greater IT community, outside of the high-performance computing, e-science niche that the project directly serves? After all, grids are either e-science or enterprise, so this couldn't possibly have anything to do with your company's IT, right?
To point to e-science and academic grids and call them niche experiments -- or to assign a higher value to one over the other -- is to turn a blind eye to the history of IT. Some of enterprise IT's most mainstream technologies today were born in academic/scientific circles.
When Sandy Lerner and Len Bosack figured out how to make different computers talk to one another across the Stanford University campus, the result was the commercial router (and the birth of networking behemoth Cisco). When major universities connected computers in Arpanet in the late '60s, little did they know the effort was an important link in the evolution toward the commercial Internet we all know today.
So it's tough to understand why grid groups have been so divisive recently, drawing a line in the sand between research/academia and enterprise -- inferring (if not saying directly) that the grid breakthroughs in e-science somehow don't match up to the rigorous requirements of enterprise grids.
In 1997, The Economist quipped that the now-late John Postel, a networking pioneer working at the Information Sciences Institute, and his academic colleagues were ill-equipped to handle the commercial evolution of the Internet. The concluding remarks of the article: "But perhaps the main lesson for the Internet is that it is time to abandon amateurism. For all their expertise, neither Mr. Postel nor most of the members of the IAHC [a predecessor to today's ICANN: Internet Corporation for Assigned Names and Numbers] have the resources (nor, indeed, the mandate) to dictate the Net's future course. Many are volunteers with full-time jobs. This was fine when the Net was largely an academic tool. Now, though, it is commercial. And commerce needs the service of professionals."
Needless to say, that particular opinion didn't sit well with the early pioneers of the Internet. And indeed, the Internet has continued to grow and prosper under this supposedly "amateur" management model.
To quote the great Yogi Berra, it's like deja vu all over again.
Can grid technology service the enterprise?
Here we are, almost 10 years later, and again, questions are being raised about research/science's ability to usher a technology developed within research and science into the supposedly more rigorous requirements of the enterprise. In some cases, the shadow of uncertainty is just an extension of an age-old rivalry between technologists in research/science and technologists in the enterprise. But in other cases, it's a malicious sort of FUD (fear, uncertainty and doubt) that's often being perpetuated by groups with their own economic interests in grid to scare potential enterprise users off of open-source (such as the Globus Toolkit) and to steer them towards proprietary grid solutions that companies can profit directly from. The common nerve they like to hit is that security, performance and quality-of-service requirements in the enterprise are higher than in research and academia.
"It's an unfair oversimplification to say that the enterprise has somehow more constraining requirements than universities do," said Charlie Catlett, TeraGrid's director and former chair of the Global Grid Forum. "We certainly are in different businesses, and we have more freedom to experiment with some things, but that is not to say that we don't also have production requirements that we have to meet just as industry does. By not focusing on trying to differentiate between research and enterprise grid requirements, and by not creating that sort of divisiveness ... we're going to make much better progress with grid."
UK gets its grid together
In the UK, e-science and enterprise grid professionals have established more of a harmony that that the US should perhaps aspire to. The Belfast e-Science Centre gets a lot of its funding from the UK's Department of Trade and Industry, and as a result, it has produced some compelling commercial-level grid projects in which e-science and enterprise grid professionals work together.
One compelling example has been the construction of a grid for the BBC. The BBC has a computing centre in most capitals of the world today -- wherever it has journalists. Its vision was that any field reporter should have access to any of BBC's compute resources [via satellite] as if he or she were sitting in the London office. So the UK Department of Trade and Industry partnered with the BBC to finance a BBC grid -- built by the Belfast e-Science Centre.
Today, the BBC's grid (and the Gridcast program) not only helps the media conglomerate deal with large file transfers -- a single hour of broadcasting can be as large as 280GB -- but also serves as an integration fabric that allows the company to tie together many different platforms and deploy software in an open manner. The open-source Globus Toolkit serves as the middleware plumbing in BBC's grid.
"The future of Belfast e-Science will be as a self-funding grid R&D entity with a number of commercial spin-out companies," said Terry Harmer, technical director at the Belfast e-Science Centre.
Enterprises get grids
So while it's become en vogue to discount e-science grids as niche experiments, the truth is that many leading organisations today are quietly harnessing the knowledge of e-science grid professionals and applying their expertise to challenging environments in enterprise. These companies aren't beating the PR drums, because publicity isn't their priority. They're interested in competitive advantage and see grid as a means to an end -- a smarter way to run their IT infrastructures.
Enterprise has only begun to tap into research/academia's grid expertise. Don't let the e-science grid FUD fool you.
Grid pioneer Ian Foster is a board member at the Globus Alliance, a vendor-neutral, non-profit-making organisation promoting the open-source Globus Toolkit in the enterprise. He can be reached at [email protected].