HP, Intel and Yahoo are teaming up over cloud computing research.
The companies are looking to advance the development and adoption of large-scale, data-intensive Internet-hosted applications and hope to foster collaboration among vendors, universities and government agencies around cloud computing, whose progress is hampered by "financial and logistical barriers," the companies said.
The three vendors are forming the Cloud Computing Test Bed, which they describe as a global, multi-data center, open-source effort designed to promote research on software, data center management and hardware for large-scale, Internet-hosted computing.
Partners in the initiative include the Infocomm Development Authority of Singapore, the University of Illinois at Urbana-Champaign, the National Science Foundation and the Karlsruhe Institute of Technology in Germany
These partners, HP Labs, Intel Research and Yahoo will host six "centres of excellence," each of which will have a cloud computing infrastructure mostly based on HP hardware and Intel processors. The centres will have 1,000 to 4,000 processor cores and are expected to be up and running later this year for selected researchers from around the globe.
Non-founding members will be invited to participate in the test bed by the end of the year, said John Manley, director of the Automated Infrastructure Lab with HP Labs in Bristol.
Manley said the research could spawn new topics for researchers to study, such as how to ensure services keep running when equipment fails, known as fault tolerance.
"It might well be that some of the fault tolerance mechanisms we know nowadays have a hard time scaling," Manley said.
When companies begin using services hosted elsewhere for their processing, security is a huge concern. Those services must be created in a secure partition that is isolated from others. The goal is to be able to provision those services quickly in their own partition or cell, Manley said.
Another concern is the management of virtual machines when several operating systems are running on one piece of hardware. The configuration can save money on infrastructure but can be more complex to manage.
Much of intellectual property that comes out of the research will be shared. "The intention is to be very open about the results," Manley said. "This is being set up as an open collaborative framework."
"Cloud computing is of critical importance to the industry and Intel is very focused on that," said Andrew Chien, vice president of Intel's Corporate Technology Group and director of Intel Research.
The project will allow universities to conduct research on large-scale environments and re-create cloud computing scenarios from the real world. It's very important to allow academic researchers to perform "empirical work at scale," as opposed to limiting them to simply theoretical explorations and small "test tube" studies, Chien said.
"The revolution of information technology computing has just begun, and the excitement around cloud [computing] is a great example of that," Chien said.
CIOs and IT buyers in general will likely see concrete benefits in a few years, when the project's research efforts yield technological breakthroughs for faster and more powerful cloud computing products and services at lower costs, said Gartner analyst Martin Reynolds.
The industry in general will also benefit from letting computer-science university students work with large-scale cloud infrastructures, since they will carry this experience to their future jobs, he said. "I think this project will be successful in attaining its goals," Reynolds said.
The centres will run the open-source, distributed computing Apache Hadoop project from the Apache Software Foundation and other open-source, distributed computing software such as Pig, a parallel programming language developed by Yahoo Research.
In a reorganisation announced in June, Yahoo announced the creation of a Cloud Computing & Data Infrastructure Group.
Prabhakar Raghavan, head of Yahoo's research unit, said that for his company, this effort builds on its M45 supercomputing data center for cloud computing research, which Carnegie Mellon University has already used. Yahoo believes academia will play a crucial part in developing the "next generation" of cloud infrastructure and web applications, he said.
Meanwhile, HP Labs will use the centres to do research on "intelligent infrastructure and dynamic cloud services," an area the company has identified as a priority, the companies said.
Intel researchers will in turn focus on extending the company's knowledge and development of processors, chipsets and other technologies for cloud computing data centres.