Kelly Russell, Cedars Project Manager

Introduction

Two and a half years ago, the Cedars project began as part of the Electronic Libraries Programme. Cedars is led by the Consortium of University Research Libraries (CURL) which represents both university and national libraries across the UK and Ireland. Research libraries have long played a part in the preservation of our scholarly heritage for other more traditional research resources. Faced with the potentially complex problems of digital preservation, CURL accepted a challenge from JISC to establish a project to look into this new area. The project is based across three partner institutions, the universities of Cambridge, Oxford and Leeds and is now well into its final year of the original JISC grant. During that time the partner sites have learned a great deal. I believe it is safe to say now, that in April of 1998, the learning curve for almost everyone involved was vertical.

In this paper I will provide an overview of the Cedars project and attempt to summarise the learning that has emerged across the project over the past couple of years. There is much to report - perhaps more than we had envisaged at the project's start. I will focus my report on the following broad areas of consideration:

• Background and Scene Setting for Cedars: April 1998
• Digital Preservation: What do we mean?
• The Cedars Project Experience
• Lessons Learned
• Next Steps

Background and Scene Setting: April 1998

To say that libraries have been influenced by new technologies in recent years is a rather generous understatement. More accurately libraries have been (and will continue to be) significantly (if not radically) changed by the introduction of electronic and network resources. This change will evolve through a two-phased ‘metamorphosis’. Clifford Lynch, Chief Executive of the US Coalition for Networked Information (CNI), suggested in 1993 that libraries were in transition from a period of "modernisation" where new technology is applied to the existing, traditional functions of the library to an era of "transformation" where applied technology changes library functions at a fundamental level. What Lynch argues is that new technology can easily automate existing library functions and services but information and communication technology can also create the conditions for innovation — conceptually this is best described as automation or renovation vs. innovation. A number of years after Lynch’s article, library services are still undergoing considerable change as they consider, experiment with and integrate emerging technology. Like other individuals and organisations, libraries too have begun to rely on many new technologies without always understanding the long-term implications of this dependence. In this context, it can be argued that libraries have yet to achieve such a "transformation".

So why is our increasing reliance on technology such a pressing issue? Over recent years libraries have grown increasingly reliant on digital materials. As the price of print materials such as scholarly journals increases, and the accessibility and functionality of digital resources becomes more attractive, libraries are investing more and more heavily in new technology. Digital resources have infrastructure and management requirements that are very different from their print counterparts. As libraries move toward what Lynch has called "transformation," their reliance on technology grows. When electronic journals were first introduced they were parallel publications (that is, they provided an electronic version of a print equivalent). These early attempts at applying new technology represented what many commentators referred to as "the horseless carriage" in that they failed to fully utilise new technology to create a truly innovative and novel product - they simply used a digital medium for the presentation of a print-based resource. For the purposes of long-term preservation and access, the paper copies could be archived and the electronic versions used just for the value-added accessibility. Many journals are still in this phase. However as digital resources begin to take advantage of technology to provide access to complex database materials, graphics, sound and video they will represent not a parallel publication but new resources themselves worthy of long-term retention and preservation. We will no longer be able to rely on a paper version without acknowledging a potentially significant loss of intellectual content.

As one reference book publisher recently suggested to me 'We don't actually have a publication anymore - we have a database.'

In the context of Cedars project work, it should be emphasised that libraries will also have to come to terms with reliance digital materials beyond simply resources within the library collection as such. Online public access catalogues, acquisitions files or administration files and even library policy material are now regularly found only in electronic form. Although the Cedars project has focused specifically on the preservation of digital collections within the library, it is important also to recognise that many of the issues also relate to the way in which libraries will need to manage their growing electronic records.

Digital preservation had been on the agenda for the JISC for a number of years before it became an integral part of the eLib Programme in 1998. As early as 1995 JISC was involved in organising an international symposium with the British Library on the preservation of digital materials. Bear in mind that 1995 saw the publication of the CPA/RLG report which was to be the seminal publication in digital preservation for a number of years to come. From the JISC/BL conference a number of recommendations emerged which were translated and scoped out as a series of studies funding under the auspices of the JISC and the National Preservation Office. During the late 1990's these studies were conducted on the following broad topics (many of which are still timely today):

• A framework of Data Types and Formats
• Responsibility for Digital Archiving and Long Term Access to Digital Data
• Post hoc rescue of digital materials
• Preservation of digital materials; policy and strategy issues for the UK
• An Investigation into the Digital Preservation needs of Universities and Research Funders
• Guidelines for digital preservation
• Comparison of methods of digital preservation

A summary report of these studies has since been produced which synthesises them down into a digestible volume.

One of the key recommendations of these studies was the need for some practical pilot projects to begin to provide some concrete experience for UK libraries in managing digital preservation issues.

Another key driver behind the desire for some practical experience was the Kenny Report (chaired by Sir Anthony Kenny) to the Department of Culture Media and Sport which recommended to the Secretary of State the establishment of a voluntary code of practice for the deposit of electronic publications to the deposit libraries. In order to ensure libraries and publishers were given an acceptable code of practice some experience was needed to back this up.

The Cedars Project - A Partnership

When the Joint Information Systems Committee issues a call for proposal in digital preservation, they were interested in collaborative proposals. For many reasons a collaboration to tackle this complex problem seemed the best approach and the Consortium of University Research Libraries, representing the main research and national libraries in the UK, seemed well placed to take on this task. CURL has 26 member libraries including each of the national and deposit libraries. The project is led by a partnership of the universities of Cambridge, Oxford and Leeds and within those partners contributions come from both the library and computing departments. Not only is the project a partnership of different research libraries, it is also, and probably more importantly a balanced collaboration between those interested in collection management and those interested in technology. The Cedars interdisciplinarity is one of the great strengths of the project.

A Shared Vocabulary and Set of Concepts

Despite the obvious advantages to the collaboration described above, the project struggled initially with how to communicate across the traditional library/computing divide (something I believe we already thought we were good at) and how to communicate about a subject new to both disciplines. Although libraries know something of preservation of materials and computer centres know a great deal about mass data storage, Cedars was perhaps the first opportunity for these groups to talk about the preservation management and long-term accessibility of a multiplicity of digital resources. The first task was to establish a vocabulary and set of concepts which would allow us to effectively communicate.

The Open Archival Information Systems (OAIS) model provided us with just such a tool. Although daunting initially, the project soon realised that this model (which was to become an ISO standard) provided not only an overall reference model for a digital archive, it also provided us with a set of terms and concepts that allowed the project to discuss and debate issues effectively. The OAIS model is discussed in more detail below.

Where does Digital Preservation Fit In?

Digital preservation refers to the series of managed activities necessary to ensure continued access to and preservation of digital materials. In order to understand what is meant by digital preservation, these managed activities must be understood on a general level. No preservation strategy is complete without a technical strategy to ensure continuing access. Broadly speaking, the three main types of technical strategy are technology preservation, emulation and data migration. Each will be considered in turn below.

Technology Preservation

Preservation of the technical environment by conserving copies of the software and specific hardware is referred to as "technology preservation". For example this might require keeping a copy of Windows 95 as well as a machine configured to run it — like maintaining your old record player to continue listening to an album collection. For some digital objects this may be the best solution — at least in the short-term — because it ensure the material is accessible by preserving the access tools as well as the object itself. However, longer term this is more problematic. For example, issues of space and maintenance of the hardware as well as costs may make this impossible in the longer term. This strategy also limits the portability of the resource since they will be dependent on hardware stored in a specific place.

Technology Emulation

There are other options however for preserving the digital resource which focus on preserving the technical environment without keeping rooms full of old hardware and software. Emulation allows for the use of current technology to mimic original hard/software in order to provide access to the digital object. Although controversial, I believe emulation potentially offers the best solution for very long-term preservation of material.

The extent to which the emulation mimics the original technical environment entirely or emulates only those components necessary to access the data remains an issue for debate. Although emulators currently exist for some of the major operating systems, in a library situation, some (if not most) of the material will be on sometimes obscure proprietary soft/hardware chosen by commercial publishers. While other organisations involved in digital archiving may have some degree of control over the types and formats of material they will accept into the archive, libraries are unlikely to be in this position. Where a funding body can make production of resources in a particular format a requirement of funding, libraries are unlikely to have much sway with most publishers who are motivated by commercial interests and not necessarily the desire for longevity of the resources.

It is also worth emphasising that an emulation strategy does not require that an emulator be stored for each archived resource. If it were necessary to store an emulator with each resource deposited this would mean that the emulators would also be dependent on a particular technical solution and require their own strategy for preservation; the costs of archiving would increase tenfold.

If emulation does not require software/hardware at the time of deposit in the archive, when is it required? It is also possible to preserve descriptive data about how the technical environment was created in the first place in order to allow for emulation later on if it is necessary. If we preserve enough information about Windows 3.1 then instead of preserving Windows 3.1 itself (or an emulation of it) we can "simply" re-engineer it again when we need it. This relies on a robust system for preserving the metadata that describes the technical environment.

To be clear: an emulation strategy means that nothing is done to the original object (it is left simply as a bytestream) and it is the environment which is re-created. The costs of emulation are as yet unknown and it is expected that the costs of re-creating complex technical environments could be astronomical. However, unlike the technology preservation model described above, the costs fall further along in the resource’s lifecycle. Instead of spending money now and for the foreseeable future, by preserving both software and hardware, emulation loads the costs at the far end. If a resource is needed in future, only then are resources required to emulate the necessary technological environment. The need for emulation is therefore determined by demand for the resource so the costs only arise if/when a resource is needed. It is worth also stressing that despite current interest in emulation, this strategy does not mean the material in the archive immediately accessible. If software engineering if required then this will mean delays in obtaining the resource. It also requires a leap of faith in terms of the power of future technologies and in the abilities of future software engineers. However of one thing we can be sure: technology will change and it will offer better, cheaper and faster solutions. Emulation may be best for resources for which the value is unknown and where future use of the material is unlikely.

Migration

Where the two options described above, focus on the environment of the object and preserving the resource through re-creating or preserving the necessary operating environment, another strategy for digital preservation is what has been called "migration". A report commissioned by the Research Libraries Group and the Commission for Preservation and Access in the US, helpfully distinguishes between migration and what has been termed refreshing" (mentioned above). The reports suggests that

"Migration is a set of organized tasks designed to achieve the periodic transfer of digital materials from one hardware/software configuration to another, or from one generation of computer technology to a subsequent generation"..

It is this last strategy in which many libraries and archives are already involved and many believe that this is the most practical approach, at least for the short and medium term. Rather than focus on the technology, this strategy tends to focus on the intellectual content and making it accessible using current technology. However for resources where it is more difficult to disentangle format from content this is not an easy option. For example, for resources like Microsoft’s Encarta or electronic journals which contain bits of sound and video the content might be inextricably linked to the format or very specific technical environment. Multiple components may require separate migration activities and this can be very complex. Indeed for some multi-media resources migration may not be possible without significant compromises in functionality. In addition, the costs of migration may, in the long run, exceed those costs necessary for preserving either the technology itself or the detailed technical specification which will allow future emulation.

Is Digital Preservation preservation or is it data management. Who within the organisation will take on responsibility for this task? One of the advantages of establishing a pilot project is that it provides a place for this work to be undertaken - particularly useful when, within an individual research library this remains very unclear. Cedars provided such a forum which had its advantages. However I should also emphasise that the project-model is only a first step (albeit, an important one) which must be followed up by integration of the experience into the existing information services at each institution. The project isn't able to do the digital preservation for the institutions. We will still need to sort out the complexities of who takes responsibility and see that it's taken care of.

Preservation of materials in libraries as a distinct area of work and study is relatively new and has come into the fore largely as a result of the "brittle books" crisis that was identified over the last few decades. Although libraries have been concerned with preserving cultural and intellectual heritage for centuries, it is only in recent years that "preservation" and "conservation" have become activities in their own right with professional associations and scholarship emerging. Book repair and manuscript conservation, for example, have grown into important areas of research and there has been a growing recognition of the importance of the resource as artefact and the historical and evidential value that may be inherent in an information object. Preservation and conservation encompass a wide range of experts working across disciplines including librarians, archivists, historians and, in relation to digital resources, computer and technology specialists. Although the preservation of material is concerned primarily with what has been created in the past, preservation specialists are often called on to predict the future in terms of how (or indeed whether) a resource might be used and in what environment. In this sense those concerned with the preservation of materials must also look far into the future to guarantee that access to material is ensured over centuries as well as back into the past to understand from the nature of its creation.

Preservation at a basic level involves the retention of information objects and their meaning. In order to realise this aim preservation must understand and, where possible, re-create the original form and function of the object to establish its authenticity, validity and evidential value. Preservation of digital materials is therefore more complex because of the dependencies which exist between the information object itself and its technical environment — an environment that is generally created using rapidly obsolete and therefore fragile technology. This dependence can be (and usually is) at both the software and hardware levels. In this sense, digital media present more complex problems than any media libraries have ever had to face. Fragile media is not a new problem: film conservationists and specialists concerned with other non-print materials have already had to face rapidly decaying media as well as abruptly changing technologies. But even these problems cannot compare with the difficulties of looking in to the digital future. This is not to say that many of the skills and knowledge of preservation of analogue materials are not applicable to the preservation of digital resources. In fact a number of the decisions involved in handling fragile materials extend beyond simply one medium. Abbey Smith of the Council for Library and Information Resources has suggested:

the skills and judgement developed in preservation professionals — the ability to discover the original form of an object and the intent of its creator, and to prolong the life of the object or return the objects as nearly as possible to its state at the time of its creation — are precisely the same skill sets that are needed for the future, albeit practised in a radically different context.

The dilemma of long-term access to and preservation of digital materials has yet to be fully recognised but even as it is currently understood, the implications are daunting. A great deal of print materials that have survived over centuries have been preserved through historical accident. The same will not be said for digital materials which rarely, if ever, have a shelf life comparable to print materials. Where a book can be left on a shelf and in 100 years time, dusted off and read, CD-ROMs will require a plan for immediate attention if they are to be accessible (and useable) after even a decade. What many fail to realise is that providing systematic back-ups and storage is not necessarily analogous to preserving the resource and the intellectual information it contains. In this sense digital objects are less like artefacts and more like "signals — signals that must be continuously refreshed or they disappear." Despite the urgency and sometimes alarming timetables for action, there is a growing pool of expertise and experience for libraries to draw on in order to develop policies for preserving digital materials.

The Cedars Project Experience

The Cedars Project was established as a response to the need for practical experience in digital preservation. The main deliverables of the project have been:

The Cedars Demonstrator Archive

Over the past 12 months the project has focused on technical developments and on the establishment of a working demonstrator archive. The demonstrator is based on an implementation of the Open Archival Information Systems (OAIS) reference model and is distributed across all three partner sites. At the "service end" it is web-based. Although the demonstrator archive was never envisaged as a fully functional service the project is now interested in how the basic demonstrator architecture might be further developed/enhanced to support the development of archive services. (see Next Steps) Over the course of the project's remaining months the demonstrator archive will continue to be tested by the following institutions:

• The University of Birmingham
• University College London
• The University of Exeter
• The British Library
• Birmingham City Library
• MIMAS (Manchester Computing)

Metadata for Digital Preservation: the Cedars Project Outline Specification

As proposed in the original bid UKOLN has played a key role in developing the Preservation Metadata: The Cedars Project Outline Specification which is one of the key deliverables of the Cedars project. This work has had both a practical focus (through the creation of a formalism for the demonstrator project) as well as a high profile international status. This was one of the first documents of its kind and has been widely recognised as an important first step for the development of a standard for preservation metadata. As a result of Cedars work in this area, the project has been invited to participate in an international working group (co-ordinated by RLG and OCLC) set up for this purpose.

Guidelines for Collection Managers

Dialogue with library and archive representatives proved early on that guidance was desperately needed if institutions were to implement a digital preservation strategy. The Cedars project has now produced a preliminary set of basic guidelines - another of the project's deliverables. This work also includes information on cost elements as well as guidance on intellectual property rights issues. However, since the start of the Cedars Project the AHDS has taken a lead in this area (through funding from the LIC) and Cedars is now interested in a strategic partnership with AHDS to collaborate in this area.

An Overview of the Cedars Demonstrator Archive

For traditional materials access and preservation were mainly one and the same and were generally done by the same organisation, even the same department. If an archive is the custodian of a set of papers they would continue to be readable and therefore useable. However for digital materials simply maintaining a bytestream does not necessarily ensure the digital material will be preserved at a level acceptable to the archive and its users. For digital materials "access" in the technical sense can be at a variety of levels and the level at which it is maintained will depend on value judgements made by the archivist and/or collection manager. In Cedars this is referred to as assessing a digital object’s "significant properties". Determining the significant properties of a digital object (i.e. the acceptable level of functionality) will dictate the amount of information or "metadata" (including detailed technical metadata) that must be stored alongside the bytestream to ensure the object is accessible to the agreed level.

A digital object’s significant properties are not assumed to be empirical; archives will make judgements at levels appropriate to fulfil their preservation responsibilities and meet the needs of the archive's user communities. For Cedars, it is the creation and maintenance of the detailed metadata associated with the object’s significant properties which is the backbone of an archive’s preservation function. The detailed descriptions and the technical systems they represent which are necessary for rendering the digital object ensure long-term preservation. In the OAIS model (described below) this information is described as representation information or, for an entire archive, a representation network. How immediate or short-term access is provided can and should be kept conceptually separate from this preservation function. Users of the archive will not have to be aware of the representation network that lies behind the archive.

Significant Properties: A simple example.If an archive takes deposit of a PDF electronic journal and decides that the significant properties are only the text within the journal, there may be no need to store information about the PDF environment but only to include information about retrieving (or rendering) an ASCII text file. These are decisions that must be made by the collection manager or archivist (often in

Significant Properties: A more complex example.An electronic journal which is published via the web as HTML . The "significant properties" are deemed to include the hypertext links (internal) as well as the multimedia functions (e.g. sound and video clips). It is at this level of functionality (full) that preservation will occur. Although end-users currently access the journal in HTML, these pages are created on the fly from SGML. For archive purposes the archive takes the SGML files. Therefore the technical metadata (or representation information) which is required includes robust technical descriptions of the objects including information about the systems and the software necessary to run the video and sound as well as less complex technical metadata about retrieving the text and images.

What is clear through work in Cedars is that decisions about the "significant properties" of a digital object have important implications for the associated metadata that must accompany the digital object to ensure it is accessible and usable over the long-term. Naturally this will have important resource implications - the more "significant properties" (i.e. functionality) deemed to be necessary the more associated metadata that will be required.

Preservation and Access

The Cedars project has developed a model for digital preservation that intentionally keeps access to archived material conceptually and technically separate from the long-term preservation function. As already mentioned above, the Open Archival Information Systems (OAIS) Reference Model provides a reference tool for the establishment of a digital archive. It provides a specific functional model of both people and systems needed to implement a digital archive. Indeed the model could also be applied to a non-digital archive. Acceptance of the OAIS model by the International Standards Organisation (ISO) is imminent and the Cedars project has provided a demonstrator project based on it. The importance of OAIS to the archiving community is undeniable but its usefulness to research libraries and archives is still largely unexplored.

The implementation of the Cedars demonstrator is a distributed archive system across Oxford, Cambridge and Leeds. Currently the Cedars demonstrator allows for data input (or ingest in OAIS terms) at all three partner institutions, storage in the archive systems at both Oxford and Leeds, and retrieval of the materials from all three sites.

Testing of the Demonstrator Archive

Initial testing of the archive was carried out this past autumn and involved representatives from each of the Cedars test sites. Participants were asked to both submit and retrieve digital materials from the demonstrator archive. Although results of the testing have yet to be analysed initial feedback suggests first that the testing provided an important learning experience for all participants to learn more about why digital preservation is important and to consider some of the skills and expertise necessary for the job.

Include some quotations here from the testing.

Conclusions: Lessons Learned from Cedars

As was suggested above, the learning curve for most involved in the project was initial vertical. Although it will be difficult to capture all of the learning that has taken place across each partner site, I will attempt to highlight the salient points.

1) A Model for a Digital Archive is Important

An accepted and reliable model for digital archives will be critical if collaboration is to be achieved. Through use of the OAIS model Cedars has discovered the benefit of adopting a shared vocabulary and set of concepts to allow implementation across a number of different local situations

2) It's All About Metadata

The metadata necessary for describing digital materials is critical but complex. Metadata of various types will be necessary (many types currently without any accepted standards or agreed formalisms) including

• Descriptive metadata (where standards have been developed and implemented e.g. MARC records)
• Administrative metadata (e.g. that which applies to preservation process itself for instance the type of technical strategy adopted)
• Technical metadata (e.g. representation information) Absolutely essential but an area where no standards exist for expressing this information
• Rights metadata to describe the (often complex) copyright information necessary for an object's preservation and use (some work has been done in this area but as yet no accepted standards have emerged)

3) Preservation and Access are not necessarily the same thing

One of the least constructive mantras used in relation to the preservation of digital material has been "you can’t have preservation without access". While it is true that there is no preservation of digital material without meaningful retrieval, it is simply misleading to imply that ACCESS to digital material is intrinsically linked to its preservation. There are important reasons (both legally and technically) why access and preservation should be considered separately. Publishers and other rights holders are often cautious about the preservation of their materials if they think unlimited public access is a necessary pre-condition. It is in the interests of preserving our scholarly heritage for future generations that we understand and discuss preservation and access in the appropriate context. We understand more now about the important difference between long-term preservation of a digital resource in an archive and the act of disseminating a copy of that digital object from the archive for access purposes. The distinction between continuing access and long-term digital preservation needs to be better understood to ensure interests in the former do not jeopardise the latter.

For digital materials "access" in the technical sense can be at a variety of levels and the level at which it is maintained will depend on value judgements made by the archivist and/or collection manager. As discussed above, this is referred to as assessing the significant properties of that object. Preservation of the byte-stream and construction of appropriate representation networks are critical to ensuring long-term preservation but technical strategies such as emulation or migration of some kind are the obvious choices for providing access over time. After a recent meeting in the UK the authors of OAIS are now planning to include a separate logical component for the preservation function - something welcomed by Cedars.

3) It isn't a Question of Black and White: Which Preservation Strategy Is the Right One?

A great deal has been discussed about the merits of using different preservation strategies - that is a technical approach to providing continuing access to the digital object. Many libraries are already engaged in migration activities with image collections to move them from one format to a newer format. Much controversy surrounds emulation despite a rather glaring lack of experience with using it for preservation purposes.

More practical work is needed to consider the effectiveness of different preservation strategies - particularly migration and emulation. Cedars has done some work (in conjunction with the CAMILEON project) comparing migration and emulation across older digital materials (already threatened with obsolescence) which suggests that both migration and emulation strategies are viable for different types of digital materials but that more work will be necessary to understand what information loss there is when each strategy is applied. If we have learned anything in this area it is that neither strategy is a panacea and that the strategy adopted for providing access to preserved resources will very much depend on the nature of the resource itself and the reason for its preservation. For both migration and emulation the jury is still out.

There are a whole host of other technical issues that Cedars has not been able to address on a practical level over the past two years. Some of these have been included on the programme and will be discussed by others with more experience than I.

4) Not Everything Should be Preserved (even if it can. . . )

Selection policies for digital materials will be critical based on considerations such as the scholarly value of the materials, the legal issues, the technical aspects etc.

5) Guidance for Collection Managers is critical

Work in this area has been ongoing both within and beyond the Cedars project. We have produced a paper on collection management guidance and are planning some collaborative to combine the work we've done with the more comprehensive work done by Maggie Jones and the AHDS about which you will hear more tomorrow.

6) Collaboration across a number of stakeholder groups is important

More work is particularly necessary with data creators and, in particular with commercial publishers. The Cedars Project experience seems to suggest that most publishers are not interested in providing long-term access to their materials (beyond their commercial viability) but may be very receptive to guidance the preservation community might provide about how they could improve the preservability of their materials through making good choices at the time of creation.

7) What are the Necessary Skills and Expertise?

We need to understand more about the skills and expertise necessary for the various aspects of digital preservation. Cedars has certainly confirmed that the level of technical knowledge necessary to construct representation networks is considerable and not likely to be found within a single library or archive.

Next Steps

The Cedars project has been granted an additional year of project funding to explore issues and provide recommendations on scaling up the demonstrator archive - perhaps to a national level. This work will help the HE library community consider use of the OAIS reference model for the establishment of distributed but integrated digital archives. In addition to this technical work, the project is also planning to collaborate with the Distributed National Electronic Resource (DNER) Programme Office to run a number of digital preservation workshops aimed at collection managers. These workshops will allow single institutions or consortia an opportunity to understand more about their roles and responsibilities for the long-term preservation of digital scholarly resources.