At the Microsoft Professional Developers Conference 2003, held in October in Los Angeles, we were introduced to the next generation of Microsoft file systems, WinFS. What does it stand for? Well, quite simply Windows File System. They can be an imaginative lot at Redmond but rest assured it will get a name. As we’re at least 18 months away from seeing it delivered, there’s no real urgency yet.

WinFS is part of Microsoft’s newest operating system, Longhorn, which has dominated parts of the media for over 2 months now. It will go on dominating pages for the foreseeable future and long after it ships. Unlike existing file systems, WinFS is designed not as a “core” element of the operating system but as an “optional” layer on top of the operating system.

Now that might sound confusing and it certainly raised a lot of eyebrows when Microsoft started to talk about Longhorn. However, Microsoft is currently portraying Longhorn as being composed of four things. Underpinning everything is the Base Operating System Services layer BOSS. This is the operating system and critical components and sits on top of the Hardware Abstraction Layer.

On top of the BOSS are three elements. The Presentation layer is codenamed Avalon. The Data layer is WinFS. The Communication layer is codename Indigo. Together these three elements are seen as the key pillars of WinFX. Microsoft has said that in theory, any of these three elements can be removed and replaced. In practice, what it is are talking about is that implementations can be removed and replaced so instead of the Microsoft GUI you could use Avalon to build a MAC style GUI.

The reality is that they will inevitably be bound to the BOSS so that you might change the appearance but not the structure.

So why do we need another file system?
The number of files that we currently store is beginning to exceed the ability of the existing file systems to effectively manage them. That doesn’t mean that the file systems are stretched beyond their limits, although some might think that. What it does mean is that the way they present and manage data to the user is becoming unusable.

When users open their folders or directories, they can find themselves faced with several hundred filenames. Wading through this can be time consuming and extremely frustrating. On one hand this makes finding data difficult and on the other, very easy to lose or delete. We often draw the analogy between the file system and a filing cabinet and it’s closer than we might think.

Many a piece of information has been thrown away due to a paperclip. We don’t have paperclips in file systems but we do have naming conventions that are difficult to work with. Look at training materials for users. We try to teach them to recognise types of files and information by the file extension and then hide commonly used file extensions in the operating system. This clearly doesn’t work and as the amount of stored data increases, needs to be overhauled.

Even understanding the naming convention doesn’t mean you can find the data. You need to know where it is stored. File paths are increasingly complex and difficult to recall. Using the operating system search options requires that you know something about the small number of identifiers we give to files. Ironically, it is easier to find information on the Internet than on most users’ computers.

So the answer to do we need a new file system is, yes, or maybe that should be, we need a better file system.

So what is WinFS?
WinFS is a place where ALL user data will live. When you upgrade a Windows XP computer all of the user data will be migrated directly into WinFS. User data is that which is stored in the My Documents, My Music and My Pictures locations. All other data will be left alone and can be moved when required.

Underpinning WinFS is NTFS. While you can have NTFS without WinFS you cannot have WinFS without NTFS. Data will not be allowed to exist in two places at the same time. It is either in WinFS or it is in NTFS. A key element here is that data will not be lost nor will developers find that their applications will no longer function.

Microsoft has recognised that it needs to ensure that backward compatibility for developers is important. To do this, it has provided full support for Win32 file API’s and existing file types. As a result, any application that needs to read/write data will be able to work within WinFS. If you use standard file formats like OLE DOC properties or media types, they are supported right out of the box.

WinFS is more than simply a superset of NTFS. It also contains some relational technology, tight links to the CLR (Common Language Runtime) and the security model. For those with security concerns, this means that security is being designed in from the start, not just for users but also for developers.