Carnegie Reporter, Vol. 2, No. 3 | The Digital Library: Its Future Has Arrived, page 3 (low bandwidth)

Preserving the Digital Library
Dr. Floyd E. Dewhirst practiced dentistry for 59 years in the same building in Los Angeles except for World War II service that included a stint drilling teeth aboard the USS Bennington. He’s retired now, but get him going and he’ll pull out his ledger book for 1938, which records the name and treatment of his very first patient. It’s right there in his own neat hand, for anyone to read.

Floyd’s daughter became a dentist too, some years before taking leave of her senses and marrying me. After dental school she went to work with her dad—but we can’t read all her records. They were kept in some special dental software and stored on floppy disks. We no longer have this software, and if we did it might not run on the computers we have today—never mind 60 years from now. Of course, by that time, the floppy disks holding the data might not be any good anyway. It’s not exactly clear how long such magnetic media, forgotten in some drawer or safe deposit box, will remain useable, but in an endurance contest lasting a century or more, hardly anyone gives it much chance against a decent grade of good old-fashioned paper.

There, in a nutshell, is one of the biggest issues facing librarians and archivists today. “All the problems associated with digital libraries are wrapped up in archiving,” says Larry Lannom, Director of Information Management Technology at the nonprofit Corporation for National Research Initiatives. “If in 100 years people can still read your article, we’ll have solved the problem.”

Perhaps a better example of what is at stake is England’s embarrassing Domesday project.

The original Domesday Book was an inventory of landholders and property compiled in 1086 by Norman monks, and today it resides safely in London. Nine hundred years later, the BBC marked the anniversary of the original by producing an innovative multimedia version providing a snapshot of life in England in the mid-1980s. The new Domesday book was vast, with more than 250,000 place names, 25,000 maps, 50,000 pictures, 3,000 data sets, 60 minutes of video and who knows how many words. Schoolchildren and researchers across the nation gathered material, and by the time it was done, something like a million people had contributed.

Unfortunately, the whole shebang was stored on a couple of 12-inch videodiscs made to be used with a British personal computer—the BBC Micro—that was once popular over there. You can imagine what happened next: the computer became obsolete and soon after that vanished from the scene. The result: 15 years after it was created, the 1986 Domesday book was basically unreadable.

The Domesday case illustrates one of the essential perils facing any digital library: the rapid rate of progress—and obsolescence—in the computer industry makes archiving digital information extraordinarily difficult. “It is only slightly facetious,” writes Jeff Rothenberg, a RAND researcher who has focused on this issue for years, “to say that digital information lasts forever—or five years, whichever comes first.”

The Domesday story has a happy ending. Information scientists at the universities of Leeds and Michigan rode to the rescue, salvaging the data and getting it into a form that could be accessed by today’s computers. But will the 1986 Domesday collection remain readable for the next 900 years? Don’t bet on it. The problem is that digital data is unintelligible to the naked eye. This article, written in Microsoft Word, is really just a series of ones and zeroes that depend on hardware and software for decoding. As Stewart Brand, noted author and online innovator puts it, “Behind every hot new working computer is a trail of bodies of extinct computers, extinct storage media, extinct applications, extinct files.”

What’s the long-term solution? Right now there is none, but a number of answers have been proposed, including migration, meaning you convert digital data to the latest format whenever necessary; encapsulation, whereby digital “objects” such as letters and videos could have some instructions attached on how to read them; and emulation, where the idea is to write software to get a newfangled system to act like an oldfangled system for purposes of running some old software.

But there are other problems, too. Take, for example, the issue of media durability. Digital media tend to be much less durable than, say, the Sumerian tablets that were created long before the birth of Christ and remain legible to this day. Estimates of the life of a CD, for instance, range from 20 to 300 years, with some cheap ones failing in a year or two. The truth is, digital media are new enough that nobody is exactly sure how long they will last, especially given variations in manufacturing, handling and even labeling.

And there are still more preservation issues. A lot of material from the past has endured because there were multiple copies all over the place, including musty trunks in people’s attics. The storage of digital data can be distributed too, as it is on the Internet, in which case preservation depends on lots of individual decisions by people who are not professional archivists. But it can also be highly centralized—after all, everyone can read the same copy—in which case it is more vulnerable to such traditional hazards as fire, flood and attack, to say nothing of hackers, hard-drive failures and excessive humidity. One centralized repository, the Library of Congress, is tackling digital preservation in a big way. Congress has agreed to give the Library $100 million to carry out a plan for collecting and preserving digital information (assuming the Library can also raise $75 million from private donors in cash, computers or other contributions).

Next page: The problem of preservation inevitably leads to the issue of ownership. In the past, libraries owned everything they offered, and preservation was their province.

Page 1 | 2 | 3 | 4