Microsoft researchers in Cambridge have developed their own version of BitTorrent which the software giant will use to distribute big files such as films, television programs and software applications over the Internet.
Codenamed Avalanche, the technology is similar to existing peer-to-peer file swapping systems, including the most famous - BitTorrent - that divide large files into many smaller pieces that are shared by all users and reassembled on your hard drive to create a copy of the original.
Such systems can scale to serve millions of users, plus reduce the bandwidth and computing costs of sending content directly from central servers. The technology has until recently been the subject of much criticism by companies complaining that copyrighted works were being shared illegally.
The problem with existing systems, according to Microsoft, is that people sometimes have to wait a long time to receive the last, "rare" pieces of a file. This is made worse when clients drop off line unexpectedly and creates bottlenecks when only a few clients have files that are in high demand.
Avalanche helps solving this problem, according to Peter Key, joint head of the systems and networking group at Microsoft Research in Cambridge, by encoding the file pieces at the server with a special algorithm before they are distributed. Each encoded piece contains information about every other piece of the original file, so users don't have to collect every last piece in order to reassemble the whole, Key said.
"Each encoded piece has the 'DNA' of all pieces in the file," another Microsoft researcher wrote. "A given encoded piece can be used by any peer in place of any piece."
When PCs in the Avalanche network receive encoded files, they randomly create new encoded files from the ones they have collected, and these are sent to other peers. When a user receives enough encoded files, they assemble them to make the original.
The system differs from BitTorrent's software in a few ways, Key said. It does not depend on central servers, called trackers, to orchestrate the download. The Avalanche client on each PC shares the files automatically among users. They do not look at other users' hard drives to find what they want. And the system works well in smaller networks, such as a corporate intranet, he said.
Plus, of course, the system is designed to prevent users from redistributing copyright material, because Avalanche will only forward files that have been signed by the publisher.
Microsoft has developed a prototype of Avalanche and is testing it by using it to distribute software applications to several thousand of its software beta testers, according to a research engineer demonstrating the software in Cambridge. The company has distributed a 4GB application in as little as a day, down from about two weeks when it sends a program directly, he said.
The software may also be interesting to TV broadcasters and movie studios. Microsoft has been in talks with both groups, and Avalanche may be introduced to users in the UK as early as next year, he said.
The BBC began testing a service last month that lets people download TV and radio programs using a P-to-P system from Kontiki. It is not looking at Avalanche currently, but will put the contract out for public tender before launching the service, said a BBC spokesman.
The concept behind Avalanche is impressive, according to Mike Thompson, principal research analyst with Butler Group, who saw the technology demonstrated. But it faces two problems of perception, he said. "Firstly, Avalanche is a mirror of P-to-P models that are coming under scrutiny for allowing illegal distribution. I believe this idea of 'good' and 'bad' P-to-P for file-sharing of copyright material will create a deal of confusion.
"Secondly, despite the 'pull' nature of the model and the security that should allow only the file to be accessed, Microsoft has had issues around security in the past - IIS being the clearest example of a secure solution that wasn't. I think the P-to-P network would be a prime target for the dissemination of viruses, despite Microsoft's assurances that it is 'safe'."
Still, Avalanche is "an excellent take on P-to-P," Thompson said.