Data storage is a very hot topic in IT these days, and I regularly get briefed by companies offering new solutions to the challenges faced by companies that produce unfathomable amounts of data daily and that are obligated to store these data for long periods while keeping much of it accessible very quickly. I've learned that solutions to data storage can affect the energy requirements of a data center and have big cost implications for an organization.
There's no two ways to slice it. Data storage is not a particularly green endeavor. It seems the best one can hope for is a reduction in the cost of storing data and improved access to it.
Hitting the delete key is one option; less data stored means lower energy requirements to store it. That's a much greener solution. Many transactions must be preserved by regulatory fiat, but certainly others can be deleted, no?
Maybe not, I'm reminded of all the movies and other art that has been lost to posterity because someone "deleted" it by re-using the tape or painting over the canvas. Why, even NASA lost data when they recycled some old magnetic tapes to store satellite data. Unfortunately the tapes had been used to record archive images of the Apollo 11 mission. Who knows what data recycling decisions we might come to regret.
And who would make these decisions? After all, history may value the historical papers of former presidents more than it will my Facebook pictures, but who knows what "virtual archeologists" and my descendants might come to value?
The storage problem has come to symbolize to me the relative newness of our data systems. We read regularly about the threats posed by vulnerability of these systems and their inefficiencies. And data storage is certainly part of these stories. But data storage also poses a threat to system reliability and information availability.
What good is all that processing power if we can't get at the information? And that is the subject of a Wall Street Journal article (August 28th) highlighting all the storage formats we have developed and basically abandoned. These include floppy disks, nine-track magnetic ape, 80-column punch cards, optical storage cards, and punched paper tapes.
As we progressed, we improved the reliability of the storage device, the accessibility of new data, and the energy efficiency of the system, and we continue to do so. But at each step, we have also lost the ability to to access data that is not very old, sometimes forever.
Mission Critical will continue to focus on the energy implications of how we run data centers, but it is also clear that we have to rethink the fundamentals of how we handle data in order to get it right. The discussion of data storage might outrank cloud computing as an issue that affects how we design, build, and run data centers.