RLG DigiNews February 15, 2003, Volume 7, Number 2, Feature Article 3

In December 2002 a group convened at the University of Leeds to demonstrate and discuss CAMiLEON's work in preserving the BBC Domesday project[1] (a social record of UK life in the 1980s[2]), which is now in danger of being lost through technological obsolescence. BBC Domesday was created to celebrate the 900th anniversary of the Domesday book of 1086, the original record of William the Conqueror's survey of England.[3]

The meeting brought together some of the original BBC Domesday videodisc developers, including Peter Armstrong from the BBC, Ecodisc's Roger Moore[4] (who brought an original glass master of one of the Domesday discs), two of the editors, and some individuals who had contributed to the content as schoolchildren; experts on digital preservation; and others interested in developing modern interfaces to the original Domesday data, as well as some nostalgic computing enthusiasts.

There were three speakers. Armstrong, as the chairman of BBC Domesday, had been heavily involved in its production. He presented several interesting anecdotes and background material about the highs and lows of the making of Domesday. Then Dr. Tom Graham, from the Consortium of University Research Libraries (CURL),[5] explained that digital preservation is our duty to future generations for both historical and technological reasons. Digital preservation needs to be understood by people at all levels, from data creators to end users, whether national, institutional, or individual. The final speaker, David Holdsworth, from CAMiLEON, has worked in Information Technology since the mid-sixties and is now an expert in digital preservation and storage. He described the choice of Domesday as a test case to demonstrate the many problems in digital preservation and how CAMiLEON has developed strategies to solve these.

There followed demonstrations of the BBC Domesday system running on the original hardware and also of CAMiLEON's modern emulation that provides an accurate reproduction of almost all the original functionality.

Armstrong, who had established the BBC's Interactive Unit to make educational multimedia, wondered if it would be possible to celebrate the 900th anniversary of the original Domesday book by producing a modern-day equivalent. It was an ambitious idea, but it captured the imagination. The plan was to give the first copy to Prince William, the "poetic successor to William the Conqueror."

Funding for the project—an estimated £2 million—was relatively easy to obtain. Multimedia was an exciting, upcoming technology, and people involved in education and national archiving, as well as computing, were keen to push it forward. The BBC put together a team of around sixty staff to develop the project and recruited pupils from over half the schools in the country to help produce the content. In all, around a million children were involved from 14,000 schools.

The map of the UK was divided into blocks, each measuring 4 x 3 km.—it is no coincidence that this is the ratio of a television screen—and each block was adopted by a school. As the UK consists of over 25,000 blocks, it was practical to cover only about half of these. It was difficult to find schools in the more-remote areas of Scotland and Wales, but the majority of England was well accounted for. Pupils investigated the land use; counted the number of doctors, post offices, and so on; and wrote articles about the people and buildings in their blocks. Each area was allotted twenty screens of BBC text and three photographs.

Back in the classroom, the school computers, with a large user base on BBC Micros, were used for data entry. The articles were sent on floppy discs to the BBC. The text was left unedited—any spelling mistakes or typing errors remained in the final print. The only alterations were prompted by the lawyers. They found that some descriptions of local characters "could cause us some problems."

Developing the hardware took two years. BBC's Interactive Unit approached Philips, the only manufacturer of videodisc players in Europe, to produce the laserdisc player. This was actually a SCSI device—the original SCSI specification had only just been confirmed—which meant a SCSI interface had to be developed for the BBC Master. The player looked like a large, slow hard disc to the computer. The BBC Master had a special read-only version of its Disc Filing System called VFS (Videodisc Filing System) and could be controlled using similar commands.[6]

The laser videodisc player produced PAL video, and the BBC Master also produced a PAL-like

video signal. The player carried a genlock and video mixing board to combine the computer and disc pictures. Other hardware was developed for the BBC Master, including a coprocessor and a trackerball. The trackerball featured three buttons, although the BBC Domesday's graphical interface made use of only two.

Logica wrote the software using BCPL, a forerunner of C. In total, over 70,000 lines of custom code were written.

"If we'd known the problems involved, we would never have attempted it," said one of the staff. The project was completed on time and on budget, thanks to the remarkable work of the team. Over 24,000 maps and 200,000 photos were processed. Remember that there were no digital copies to work from—the paper originals of the maps were quite literally "cut and pasted" together. Each map and photo was captured as a single frame of continuous videotape. These then had to be captioned and have their copyright cleared. As well, over 8,000 data sets (traffic congestion, radiation levels, etc.) were stored.

The size of the Domesday project was overwhelming. The budget of £2 million sounds like a lot, but the real cost must have been far, far more than that when the dedicated work of all the schoolchildren and volunteers is considered. It has been estimated that if you worked a forty-hour week viewing Domesday, it would take seven years to see all the information. One source calculated that it would have cost a quarter-million pounds for institutions to access that amount of data, which made the price tag of the Domesday system sound like a bargain.

When the plans for the project were announced, the estimated price was £1,100, but when Domesday came to market, it had increased to over £4,000. As this was too expensive for most libraries and schools, Domesday became a commercial flop. The first set of discs was presented to the keeper of records at the Public Record Office, to be placed alongside the original Domesday book.

Life went on. The BBC Interactive Unit developed a few other ideas but eventually folded when the director general decided there was no future in multimedia. Armstrong and a group of colleagues bought out the department and set up the MultiMedia Corporation. Reworkings of the Domesday ideas appeared in other forms: the 3D World Atlas (Domesday on a global scale) sold over a million copies; Oneworld.net [7] features Another Domesday, which focuses on global justice issues and is one of Kofi Annan's favorite Web sites. BBC Domesday became an icon, the granddaddy of interactive multimedia. And then it became obsolete.

The CAMiLEON project (Creative Archiving at Michigan and Leeds Emulating the Old on the New) has spent three years developing strategies for digital preservation and testing them with materials such as the BBC Domesday system. The BBC Domesday project encapsulated many difficult problems encountered by those working in the field: a huge amount of multimedia data, technological complexities, and the intellectual property rights (IPR) issues.

There are several aspects to preserving BBC Domesday. First is the decay of the media—discs get scratched during use and become less reliable. The hardware to read the discs is rare, and the few remaining laserdisc players are prone to break down (and require very specialized repair). All the hardware is long past its shelf life. BBC computers have always been durable, but not many were produced with the special Domesday extras. The Domesday system also has a particular look and feel that requires preservation in addition to the actual content.

CAMiLEON obtained access to a semi-working Domesday system donated by the School of Geography at the University of Leeds. One of the first tasks of preservation was to transfer the data files from the twelve-inch laserdiscs to modern hardware, storing the bytestreams in a media-neutral form. A Linux PC could be connected to the laserdisc player using a SCSI cable, allowing the PC to read the text articles and database. Images, including still-frame video, were transferred to a PC using a standard video frame-grabber card at maximum resolution. These images were stored in an uncompressed format to avoid quality loss or the introduction of artifacts (as can occur with JPEG compression). In total, around 70GB of image data was transferred per side of each laserdisc.

The next step was to develop software that emulates the adapted BBC Master computer and the laserdisc player on which the original BBC Domesday system ran. An open-source emulator—BeebEm[8]—was used as the starting point for this software. Emulation of the specific Domesday system hardware had to be incorporated by CAMiLEON, which included the coprocessor, SCSI communication, and the many functions of the laserdisc player.

CAMiLEON's philosophy is to preserve the data in its original, unmodified format (i.e., the original abstract bytestream, not in the same physical medium). Software can then be written to use this data: perhaps an emulation of the original system, perhaps a tool that reformats it into a modern format, or perhaps software that provides a new interface to the data. This view builds on the ideas of the CEDARS project.[9] For BBC Domesday CAMiLEON developed an emulation of the original system in which knowledge of how the original system worked is encapsulated. The emulation software, together with the abstracted data, provides a record of the original BBC Domesday system. A "black box" emulation of the laserdisc player was written to allow the emulated BBC Master to access the data recovered from the original laserdiscs.

To avoid the problem of emulation software becoming obsolete, it was important to ensure that the software was not chained to any specific operating system or machine architecture. Careful development with a clear focus on the goal of longevity will make it easier to run this software on a future (as yet unknown) computer, needing only a few simple (and well documented) modifications.[10] This also means that it should be possible to port the emulation software to any current machine.

Currently the CAMiLEON BBC Domesday emulator runs only on Windows because, owing to time constraints, a Windows-based emulator, BeebEm, was used as the starting point. Because this was not written to follow guidelines for software longevity, it is tied to the Windows platform. The CAMiLEON team is currently seeking a small amount of funding to complete the software-longevity work and prepare the emulator for archiving.

Sadly, it is unlikely that Domesday will become available to the general public unless the IPR problems can be solved. The contents of the discs are heavily tied up in copyright—parts are owned by the BBC, the Ordnance Survey, and possibly the Local Education Authorities and schools. However, it may be possible for owners of original BBC Domesday laserdiscs to gain access to the preserved data and to make the emulator software publicly available. This would allow access in library reading rooms and some schools, for example. CAMiLEON is interested in examining and solving Domesday's IPR issues. Andrew Charlesworth discusses the issues in detail in "Legal Issues Arising from the Work Aiming to Preserve Elements of the Interactive Multimedia Work Entitled 'The BBC Domesday Project'"[11].

The recent auction of a BBC Domesday system on eBay is evidence of the revived interest in the project. There are also a couple of people working to produce modern interfaces to the original Domesday data—they met each other for the first time at the CAMiLEON meeting.

CAMiLEON is keen to hear from any participants who worked on, or contributed information toward, BBC Domesday and would be interested in their views on the issue of making it available to the public. Please contact Paul Wheatley if you can help.

RLG DigiNews (ISSN 1093-5371) is a Web-based newsletter conceived by the RLG preservation community and developed to serve a broad readership around the world. It is produced by staff in the Department of Research, Cornell University Library, in consultation with RLG and is published six times a year at www.rlg.org.

Materials in RLG DigiNews are subject to copyright and other proprietary rights. Permission is hereby given to use material found here for research purposes or private study. When citing RLG DigiNews, include the article title and author referenced plus "RLG DigiNews,

." Any uses other than for research or private study require written permission from RLG and/or the author of the article. To receive this, and prior to using RLG DigiNews contents in any presentations or materials you share with others, please contact Jennifer Hartzell (jlh@notes.rlg.org), RLG Corporate Communications.

Please send comments and questions about this or other issues to the RLG DigiNews editors.

Co-Editors: Anne R. Kenney and Nancy Y. McGovern; Associate Editor: Robin Dale (RLG); Technical Researcher: Richard Entlich; Contributor: Erica Olsen; Copy Editor: Martha Crowe; Production Coordinator: Carla DeMello; Assistant: Valerie Jacoski.

All links in this issue were confirmed accurate as of April 15, 2003.


		RLG DigiNews		BROWSE ISSUES		SEARCH		RLG