With 15 hospitals, $2 billion in annual revenues, and 12,000 employees, Baptist Memorial Health Care is one of the largest not-for¬profit organizations in the United States.
Because Baptist has experienced such explosive growth, its data storage needs have grown dramatically from 2 TB in 2002 to more than 138 TB in 2005. Part of this growth is due to recent federal regulations, which mandate that hospitals must retain and be able to access patient data growing at a rate of more than 300 percent a year for up to 21 years. In addition, this patient data involves much larger files. Important new medical technologies, such as CAT scans that create 3D images of patients internal organs, require extreme bandwidth, massive storage capacity, and near instant retrievability. Baptist needed a cost¬effective solution to manage its exponential data growth.
Managing sheer data volume was not the only challenge under consideration. Because there are more than 45,000 patients a year under Baptists care, the IT department also had to factor in the link between effective data management and the quality of medical service. In health care, if you lose data, it could affect patient care, explains Hal Weiss, Systems Engineer for Baptist Memorial Health Care. On the flip side, if doctors have instant access to the images and data being generated by todays medical applications, that also impacts the patients health in a very positive, dynamic way.
Before beginning its disk¬based backup and restore initiative, Baptists IT department backed up solely to tape libraries. As the organization expanded, their two key limitations with tape unreliable and slow backups became readily apparent. So did the need for a fast, reliable alternative to tape for disaster recovery protection and long¬term data archive storage, in addition to backup/recovery.
As Baptist Memorial Health Cares data storage requirements continued to grow, backup windows were not being met, backups often failed, and critical patient data could not be accessed quickly enough. To address these challenges, Baptist recently implemented dramatic upgrades to its backup/restore and archiving infrastructure by installing a COPAN Systems Revolution 200T Maid (Massive Array of Idle Disks) -based storage system.
Weiss knew that tape was no longer a viable solution for protecting mission-critical data. You simply cannot back up 138 terabytes to tape. It doesnt matter how many tape drives you have. There just isnt enough time, he says.
With a huge patient and employee database, Baptist backed up 4.5TB every day through approximately 2,500 separate backup jobs to five tape libraries. However, because the tape was fragile, unreliable, and slow, it could potentially put the restoration of Baptists imaging, patient, and billing data at risk. Specifically, Baptist faced the following challenges:
Its 960 GB Exchange server could not meet backup windows, taking 6.5 hours per day and suffering multiple failures;
A critical 9 TB image data server, with 386 million 55K files, took 30 days to back up data and did not meet Baptists standards;
Baptists data center, which had been designed to last five years, was reaching capacity much sooner than was forecasted and would be out of space in less than three years; and
SAN-based storage was increasing by more than 5 percent monthly from 108 TB to 138 TB in just 5 months and the backup failure rate had reached 32 percent, missing backup windows 23 percent of the time.
We were definitely struggling to keep up with our growing storage needs using traditional methods, Weiss explains. I no longer had any kind of life outside of work, and I was not the only one. It was clear we had to do something.
As Baptist searched for a solution, the organizations technology experts considered critical requirements:
Higher performance backing up 5 TB in 18 hours;
Increased reliability eliminating failed backups;
Improved data availability facilitating quicker restores via online archives;
Lower cost roughly the same capital cost per/GB as tape; and
High-density storage requiring less floor space than alternatives.
Baptist reviewed various storage products that could help the company protect its data by moving to a disk-¬based approach. COPAN Systems Revolution 200T emerged as the leader because it was the fastest, most reliable, and most cost-effective answer for both backup/recovery and long¬term data accessibility. Also, COPANs product offered the highest capacity in the smallest footprint, saving valuable data center space.
Weiss installed COPANs Revolution 200T, with its MAID system, on an evaluation basis to determine if it was the right fit for Baptists needs. Were very conservative in that we cant afford to lose any data or have any degree of data corruption, explains Weiss.
On the other hand, diagnostic technology is moving ahead so rapidly, and the regulatory requirements surrounding the health care industry have become so stringent that we needed to look at the latest technology in order to keep up. Just as were aggressive in adopting new diagnostic technology to improve patient services, we also have to be aggressive in implementing storage technologies to back up this data.
MAID technology is a storage system in which an array of disk drives can be powered up individually or in groups, and used only when needed to preserve drive life and lower costs due to more effective resource utilization. COPANs product out-performed Baptists previous tape solution in every measure from bandwidth and speed to overall cost. In fact, the first backup was performed in just over 3 hours, half the previous time of 6 hours.
Baptist was so pleased with its results from the Revolution 200T that it purchased the system just 21 days into a 60¬day demo/evaluation program. I was floored by its performance, and also by how easy it was to install and begin using it immediately. We unwrapped the Revolution 200T, plugged it in, fired it up, and it ran! We did our first successful backup within hours of installing the system, Weiss said.
The ongoing result is that the Revolution 200T provides Baptist with the reliability, speed, and responsiveness of disk, along with the scalability and affordability of tape. It also gives Baptist consistently fast and reliable backup/recovery and archiving access for its mission-¬critical data and patient records including billing data, clinical documentation, and radiology imaging.
Return on Investment
Weiss says he expects to see a return on investment in two years, based on reduced expenses in tape media alone. He says Baptist is saving additional money by not having to move tape offsite, and eliminating the labor and materials expenses associated with tape. Utility costs have also been slashed. Add up all the hard savings, and Weiss estimates that the Revolution 200T paid for itself in about six months.
Perhaps the biggest cost savings of all comes from the smaller footprint required by a Revolution 200T, compared with any other storage solution. Since installing its first Revolution 200T, Baptist determined that its current data center will not need to be expanded or replaced nearly as soon as had been forecast.
Baptist looks forward to:
Implementing four more shelves in its current unit;
Saving $85,000/year by not moving tape offsite; and
Saving $45,000/year in reduced labor and materials associated with tape.
Its not just backup/restore
Peoples lives will be affected by how well this works, Weiss says. We are also saving brain cells, he adds. This is a technology that we do not need to think about. It simply does its job. The stress of not knowing whether backups/restores will work is gone. Our backup failure rate has gone from more than 30% to absolute zero, and we have not had a backup/restore failure since installation.
The stress of wondering how we will keep up with our data demand is gone. A year ago, we did not have a strategy for keeping up with our data storage needs. Now, we have a strategy that we know will work going forward.
The Revolution 200T enables Baptist to improve patient care because it gives physicians near ¬instantaneous access to patient records. Data-intensive organizations like ours have to be willing to embrace new technology or be left behind, Weiss adds.
Working with COPAN Systems, Baptist can now continue to meet strict governmental and health care data protection regulations, such as the Health Insurance Portability and Accountability Act (HIPAA) and Sarbanes-Oxley Act (SOX). COPAN Systems easily met Baptists strict data retention and protection requirements. For Baptist, data security, integrity, and accessibility are invaluable because the loss of patient data not only risks severe financial and legal costs, but it can also affect patient care.