Research Agendas Set Course for Digital Archiving and Long-Term Preservation

Margaret Hedstrom
School of Information, University of Michigan

Two recently released reports, It’s About Time[1](pdf) and Invest to Save[2], propose complementary research agendas for digital archiving and long-term preservation. It’s About Time presents the findings and recommendations of a workshop co-sponsored by the Library of Congress, National Digital Information Infrastructure and Preservation Program, and the National Science Foundation’s Digital Government and Digital Library programs. Invest to Save reports on the recommendations of a working group of North American and European experts that was co-sponsored by NSF and DELOS, Network of Excellence on Digital Libraries, funded by the European Commission's Information Society Technologies Fifth Framework Program.[3]

Both reports stress the growing centrality of digital information in government, commerce, research and education, cultural heritage, and even interpersonal communications, as well as the inadequacy of current digital preservation strategies and methods to address challenges posed by increasingly complex digital entities. As the title suggests, It’s About Time identifies concern with “the long term” as one characteristic that distinguishes digital preservation research from research on digital libraries or storage technologies. The long-term perspective raises issues of technological obsolescence and evolving technologies, social and managerial concerns over the threat of interruptions in management of digital archives, and economic questions around the business and funding models needed to sustain digital archives over many generations. The report outlines priority areas for research intended to develop partnerships between academic researchers, researchers in the information technology sector, and program managers in government agencies who have responsibilities for managing and preserving large data collections.

It’s About Time defines research challenges in four broad categories: 1) technical architectures for archival repositories, 2) attributes of archival collections, 3) digital archiving tools and technologies, and 4) organizational, economic, and policy issues. Despite important recent developments in technical architectures, considerable work is needed to define a spectrum of repository types and services that will be capable of preserving complex and diverse digital content. There is also a need for further elaboration of abstract repository models, implementation, and evaluation, especially with regard to scalability and cost. The report recognizes that human labor is the most costly element in digital preservation and one that is likely to increase, while storage and processing costs continue to decline. Therefore, there is a premium on developing methods that reduce the amount of human intervention in digital archiving processes. Research on the attributes of archival collections would address issues such as models for curatorial processes that might lead to automating portions of them, better formal methods for characterizing complex digital entities and collections, methods to manage collections that are built and preserved jointly by several different institutions, and decision models to support curators’ choices of appropriate preservation strategies and methods. New preservation tools and technologies are needed to support automated handling of digital archiving processes during acquisition and ingest, for naming and authorization, and for managing evolving technology components. Finally, the report calls for more attention to metrics to measure the costs, benefits, and values of digital objects and to evaluate the costs, performance, and effectiveness of various digital preservation strategies across different types of media and content.

Invest to Save echoes many of the themes in It’s About Time, although the latter report makes an even stronger case that archiving processes designed for the paper and analog world need substantial re-conceptualization and re-design to preserve digital information effectively. The report also identifies new areas for research and development. Under the rubric of Preservation Strategies: Emerging Research Domains, the report calls for further elaboration of current models for archival repositories, along with development of three new types of repositories for formats and format documentation, software and software documentation, and peripheral devices. New research domains also include such areas as archival media, better methods for salvage and rescue of data in obsolete formats, models to describe the functionality and behavior of digital entities, and the development of context-aware digital entities that could respond to risks in their environment. The section of the research agenda on Re-Engineering Preservation Processes recommends research on new models for preservation processes followed by development of tools that would support automation of many curatorial processes. In the area of preservation tools and technologies, Invest to Save also places considerable emphasis on preservation of complex digital entities, tools for automatic metadata generation, methods for re-purposing digital content, and the challenge of multi-lingualism in digital archives.

These two reports offer plenty of suggestions for projects that will keep digital preservation researchers and curators of digital collections occupied for several years. There are already indications that some of the sponsors are soliciting proposals in this area. The recent call for proposals from the NSF Digital Government Program includes digital archiving as one key component. The European Commission will likely include digital archiving projects in its Information Society Sixth Framework with the possibility of more international projects.

Notes

[1]It’s About Time: Research Challenges in Digital Archiving and Long-Term Preservation, Final Report, Workshop on Research Challenges in Digital Archiving and Long-Term Preservation, April 12-13, 2002, sponsored by the National Science Foundation, Digital Government Program and Digital Libraries Program, Directorate for Computing and Information Sciences and Engineering, and the Library of Congress, National Digital Information Infrastructure and Preservation Program, August 2003. (back)

[2]Invest to Save: Report and Recommendations of the NSF-DELOS Working Group on Digital Archiving and Preservation, prepared for the National Science Foundation’s (NSF) Digital Library Initiative and the European Union under the Fifth Framework Programme by the Network of Excellence for Digital Libraries (DELOS), 2003. (back)

[3]NSF and DELOS sponsored seven additional joint working groups on topics of interest to the digital library community. All the working groups’ reports are available. (back)