The Coal Authority was established in Britain by Parliament in 1994 to undertake specific statutory responsibilities associated with the licensing of coal mining operations in Britain; dealing with subsidence damage claims which are not the responsibility of licensed coalmine operators, with property and historic liability issues, such as the treatment of mine-water discharges and providing public access to information on past and present coal mining operations.
The reason the Authority is required to provide public access to information regarding past and present coal mining operations goes back to the Coal Mining Act of 1872 requiring coal mine operators to deposit coal mine abandonment plans, when colliery's closed, to HM Inspector of Mines and Quarries.
The Coal Authority maintains and manages mining maps, plans, historic photos and documents from the United Kingdom dating back prior to 1872. Not only is this archive of documents a public collection, it's a national historic record as well. In an attempt to preserve and protect this vast collection, the Authority has set upon the task of digitising all these records to protect and preserve them and provide a digital database for public access.
Digitising maps and papers
This vast collection of documents includes over 100,000 maps and paper records. Most of these maps are larger than three feet square, and some are as large as 6' X 9'. For the smaller documents, scans are easily managed with three 60" roller scanners, each controlled by a single PC.
In order to accommodate the extremely large maps, the Authority worked with DigiData, a development by Bournemouth University, to devise a system to scan sections of the map and stitch the sections together with software in order to build a single image. DigiData designed a system utilising six very high-resolution scan-back cameras arranged on a gantry over a large glass-covered bed, on which the large-scale maps can be placed. The cameras feed very high-resolution images into six imaging servers, which in turn feed each separate image into a single server.
Here custom software then stitches each partial image into a single complete image. When completed, it is fed into the database as a single digital image of the original map. Special care is taken to ensure that the scale and stitching is accurate and seamless and all handwritten annotations are legible.
After two and a half years, the Authority is about two thirds complete with the collection of smaller maps and is beginning to scan the larger maps. The large maps create a digital image typically between nine and twelve gigabytes and currently the system is managing about 70 terabytes of data. Once complete the database is estimated to grow to about 110 terabytes for just the map images.
Enter UDO optical archiving
The database contents are managed with QStar HSM software. QStar provides a cache management system for the imaging software and manages the data from the imaging application on a hard drive cache and then secures it to one of two storage libraries on site.
Originally the Authority selected magneto optical technology to store the data, however at only 9.1 gigabytes per disk, the Authority was quick to identify the need for more storage. In its search it looked at both tape and magnetic disk technology, however disk required a disaster recovery and backup strategy that proved to be too expensive and tape did not have the longevity or random access required for his application.
The Authority selected UDO (Ultra Density Optical) technology from Plasmon because it provided more than three times the capacity of magneto optical disk and the media platters were less expensive. They were also pleased to discover that these library systems could be retrofitted with the new technology and with a software upgrade. QStar could also support the new technology. In this way, there was no requirement to learn new software and the Authority was able to keep capital costs down by simply integrating the new drive technology into the existing systems. Currently, the contents of the Authority database are stored on a mix of both magneto optical and UDO platters.
As well as having the digital collection available for public viewing at its offices in Mansfield, the Authority plans to make the entire collection available online. To do this, it is creating a separate database with lower resolution JPEG format images managed also in a UDO library controlled by QStar HSM software. In this way the public will, eventually, be able to access the mining information from the Internet, saving it the need to travel to the Authority offices in Mansfield, Nottinghamshire.
In a recent interview David Brown, the Authority's Network Manager, and responsible for developing the storage systems required for the project, told us: "Because of the size and volume of the images, we could not have made it work without QStar's software. They were the only solution that could handle the requirements we had."
The storage system runs on Sun Solaris with Sun V240 and V250 servers. The front-end imaging and viewing application is Windows-based NVision software from OCE. The high-resolution images of the large maps are processed on Apple Power Mac G5s prior to being transferred to the Solaris based storage management systems
Find your next job with techworld jobs