Developing a 3D Digital Library for Spatial Data:
Issues Identified and Description of Prototype

	Figure 1. Polygonal mesh model of Native American ceramic vessel with regions of curvature identified by color

Overview

The increasing power of computing techniques to model complex geometry and to compare models to identify similarities among them has created powerful new capabilities to analyze and interact with data representing three-dimensional (3D) objects. The techniques to model and extract meaning from 3D information create complex data that must be described, stored, and displayed to be useful to researchers. Because two-dimensional (2D) data representations afford a limited view to scientists in related disciplines, the Partnership for Research in Spatial Modeling (PRISM) project at Arizona State University (ASU) developed modeling and analytic tools that raise the level of abstraction and add semantic value to 3D data.

The goals of the project have been to improve scientific communication and assist in generating new knowledge, particularly about natural objects, whose asymmetry makes study using 2D representations insufficient. The tools developed use curvature and topology to help researchers understand and interact with 3D data, thus simplifying the analysis of surface and volume in the representation of an object. The tools automatically extract information about features and regions of interest to researchers; calculate quantifiable, replicable metric data; and generate metadata about the object being studied. To make this information useful to researchers, the project developed prototype interactive, sketch-based interfaces that permit researchers to remotely search, identify, and interact with the detailed, highly accurate 3D models of the objects. The results support comparative analysis of contextual and spatial information and extend research on asymmetric man-made and natural objects.

Digital libraries are in the midst of a significant evolutionary process. Ever since the initial efforts to apply computers to library catalogs and circulation control, the scope and complexity of the technology-assisted interactions between libraries and their users have created both challenges and opportunities. As catalogs were computerized and the Internet began to link libraries, the emphasis of projects shifted to creating linkages between catalogs to let users search across multiple libraries.

Developing standards for communication, search, and display were initial challenges that, once addressed, gave researchers access to library collections around the world. Soon, modem access permitted researchers using home or office computers to search as effectively as at the dedicated terminals in the physical library. Projects to provide internal access to graphics and images of objects within collections were undertaken using stand-alone applications like HyperCard or Macromedia Director, which stored content on videodiscs or CD-ROM.

As graphic browsers became more powerful, remote access to images became possible. Faster connections, more-powerful hardware and software, and cheaper storage fostered significant projects to digitize graphic content and expanded their scope even further to create digital virtual collections. The interpretive material woven into the primary source material provided by digital libraries continued to increase tremendously, as did access to collateral material available via the Web. As computing power grew, objects could be captured and displayed on the Web.

Scientific visualization has followed a similar path, from computer-assisted statistical analysis to the sophisticated modeling and visualization of complex scientific data that have dramatically changed the kit of research tools in virtually every field. Data can be acquired as textual or numeric descriptions, as well as optically using cameras or microscopes, or electronically using sensors or CCDs. Two examples of the many scientific visualization projects displaying images of 3D objects include:

• Forma Urbis Romae project at Stanford (FURStanford), which uses a viewer to display 3D images of the fragments of a Roman artifact
• digital morphology project at the University of Texas (DigiMorph), which displays surface-modeled images derived from computer-aided tomography (CAT) scan data of biological specimens

Figure 2. Screen capture, Forma Urbis Romae project

Figure 3. Screen capture of DigiMorph, the digital morphology project at the University of Texas

Projects such as these permit users to zoom, rotate, and visually interact with the image using their keyboard and mouse. The DigiMorph project also provides access to the individual-slice images that are 2D representations of the interior composition of CAT-scanned specimens.

2D vs. 3D Object Representations

Unfortunately, most digital libraries to date use two-dimensional images to represent 3D objects. Single slices of CAT scan or confocal microscope data can provide valuable information about the interior of the object, and QuickTime VR views of exterior surfaces let researchers view detailed pictures of object surfaces. Though these images of 3D objects display surface attributes and can support detailed visual analysis and comparisons of surface models, the image representations lack key components essential for many types of scientific analysis. These 2D image models do not capture the underlying geometry and topology that define the relationships between the points that comprise the surface or interior components of the object. This spatial data about the geometry and topology associated with the object is necessary for comprehensive modeling, measurement, and analysis of the 3D objects.

More complex objects in terms of variety in shape and changes in curvature are more difficult to quantify and analyze. By developing mathematical techniques to represent the shape and curvature, accurate models of the surface of 3D objects, such as ceramic vessels, bones, or lithics involved in the PRISM pilot projects, can be created. The surface models and sophisticated mathematical tools provide the ability to analyze, identify, and compare the objects that they represent.

The precision of the models supports measurements whose accuracy equals or exceeds that possible using traditional 2D tools such as calipers and rulers. For example, one of the scanners captures surface data points less than 300 microns (0.3mm) apart, producing high-density triangular meshes with an average resolution of over 1000 points per cm2. In addition, measurements such as height, width, maximum height or width, surface area, or volume can be easily, consistently, and accurately calculated from the scanned data using software tools developed by the project team, even for asymmetric natural objects.

Figure 4. Conceptual model of PRISM digital library for 3D spatial data

Use of 3D data also makes possible new measures based on topology and global or local changes in curvature that define the shape of the original object. Using mathematical models and surface and volume information, many new and powerful analytic tools become available—boundaries can be objectively identified, small local areas of changes in curvature identified and compared, and accurate, replicable measurements calculated automatically.

Access to the object geometry adds the capacity to objectively quantify and analyze spatial measures that define the object. Such geometric information permits analysis of the object using spatial descriptive characteristics such as:

• area of features or overall surface of the object
• volume within the object or its internal components
• orientation of internal components and of the overall object
• proximity and distance between points of interest
• changes in local or overall curvature
• object symmetry

These objective measures augment subjective descriptions and are extremely helpful to domain researchers, particularly when studying asymmetrical objects made by man or occurring in nature.

Features

Once meaning has been linked to the changes in topology, shape, or curvature by the domain scientists, a meaningful "feature" can be defined. The modeling process can provide an objective method to calculate physical measurements and to identify boundaries and local areas of interest to researchers, by the changes that are associated with the feature. Once identified, each feature can be described by its size, position, shape, or curvature.  Examples of features that can be extracted from the model data include the maximum diameter or height of a ceramic vessel.

Figure 5. Mathematically extracted features: bone surfaces (left), lithic stone tools (right)

Features that are mathematically abstract can also be of interest to the researcher, such as the base or neck of a vessel, the keel of a ship, boundaries of the joint surfaces on a bone, or spindles that form in the nucleus of a cell during meiosis. Often the tools developed to identify features and regions offer additional capabilities that raise new research questions within the disciplines. For example, ceramic analysts have used tools that identify mathematically defined features found on the vertical profile curve of a vessel such as end points, points of vertical tangency, inflection points, and corner points. These features are extremely helpful in analyzing abstract concepts such as vessel shape and style.

Ideally, fresh research questions arise for the domain scientists as each new tool is applied to the object data, which raises new challenges for the computer scientists and fosters another cycle of new tool development. In addition to the tangible research benefit, a significant result of this process has been the cross-pollination of graduate students and a considerable increase in collaboration among faculty researchers across disciplines.

The PRISM Project

To begin to address these issues, PRISM at Arizona State University has developed prototype digital collections of scanned data describing 3D objects. The scanned data includes descriptions of geometric and topologic data in addition to the object surface. Goals of the project were couched in terms of developing partnerships between computer scientists and domain researchers to develop processes to:

• create quantifiable, measurable models
• automatically identify and extract data
• create catalog information
• automatically populate databases
• support analysis and interaction
• help answer research questions and generate new knowledge

The project grew from an interdisciplinary team of researchers from Computer Science, Mechanical Engineering, Anthropology, Fine Arts, and Information Technology. Two lab areas were used, one proximal to the computer science researchers, the other adjacent to the domain science departments. Additional research partners created a web of physical resources and personnel across the university that encouraged interaction and team-based development.

Initial research questions posed by a discipline scientist were shared at team meetings. Research and exploration were initiated, potential solutions brainstormed, and development tasks assigned to smaller teams. As further discussion was needed, prototypes were developed, evaluation input was sought, and formal and informal activity within the team moved the tools and techniques forward. Formally, presentations at team meetings were shared discussions. Informally, after the initial team formation, when sharing expertise and approaches was sufficient, the team members would initiate ad hoc group interaction to work through problems, share ideas and solutions, and compare progress in other project areas.

A summary of the project development sequence includes:

• team formation and process development
• shared vocabulary development
• XML schema and DTD development
• modeling and visualization research and development
• data acquisition—scanning/digitizing the objects
• data extraction—using the tools to extract features, identify regions of interest, and create metadata tags
• data storage
  • text and tabular data
    • links to other databases
    • generated or derived data
  • binary geometric/spatial data
    • point cloud data
    • polygonal mesh
    • surface/volume models
• query interface development
• evaluation
• revision/identification of new research questions

Figure 6. Prototype profile-based visual query interface for searching ceramic vessels

Metadata

Metadata to describe the object—and the modeled and derived representations and measures—was an important project design issue. A conceptual goal of the metadata component of the project was to develop an extensible schema structure that could accommodate the addition of new types of objects as the project evolved. An object class was defined as the master class document type definition (DTD) for each item in the digital library database. For the 3D digital library project, the additional descriptive data about each object was defined and organized as classes of contextual or spatial data definitions.

Contextual definitions describe text and metric information about the object. This class includes subclasses for metadata such as type, item name, catalog number, collection, or provenance that are associated with objects as they are acquired, processed, and archived. These fields were initially determined by existing descriptive data elements, though efforts were made to design a schema structure that would accommodate other object types. Several design iterations to refine the schema so it would work across object types have been completed.

Spatial data types define the 3D attributes of the object, including raw data, thumbnails, models, and calculated and derived data about the topology, shape, and composition of the object. Common descriptive components and geometric elements permit shared use of the modeling and analysis tools across classes of objects as new object types are added. An additional goal was to develop standards for description and organization that permit automated cataloging and population of databases as objects are scanned and processed.

Because of its familiarity and the availability of resources, an SQL database was used to store the contextual and spatial data for the initial project. Fields were assigned to each data element and metadata description.  The large spatial data files were stored as hyperlinks to data storage databases. Generally accepted data formats such as binary, PLY, HTML, and XML, were used to make data accessible and simplify migration and access to the data over time.

The process of acquiring object data from the ceramic vessel, bone, and lithic pilot projects starts with laser scanning to capture the 3D data that defines the object. Cellular data is obtained from a confocal microscope. Other data from digital cameras, CAT scanners, MRI, and satellite and aerial scanning has also been used. Once the point cloud data has been obtained from assembling the scanned data, mathematical modeling is applied to identify features and regions of interest to the domain scientists. Software tools developed by the project team generate analytic data about the original object, automatically assign metadata about spatial characteristics, and populate the database.

A visual query process was developed to permit researchers to search and interact with the data using both contextual (text and numeric descriptive data) and spatial (shape and topological attribute) data. A sketch-based interface was developed that permits users to input both context and sketches to visually describe the object to initiate the search. Several text and spatial matching algorithms are used to identify and rank order objects in the database that match the search criteria.

Figure 7. Query interface and search process diagram for PRISM ceramic vessel data (Click on image for larger display.)

Initial development of the digital collections focused on Classic Period (A.D. 1250-1450) prehistoric Hohokam ceramic vessels from central Arizona housed at the Archeological Research Institute at ASU. Additional development has involved bone shape and surface, lithic tools, brain structures, and DNA structures in fertilized mouse egg cells. Research has extended to other disciplines with interest in spatial analysis, including cloud formation, wind erosion, and facial recognition.

We feel that one of the next key challenges in digital library development is to create the processes and tools to support interaction with 3D geometrically and topologically rich information about 3D objects. Capturing the breadth and complexity of data that spatially defines 3D objects offers many design and process challenges in developing digital collections. Several important issues regarding standards and conventions must be addressed to create geometrically accurate 3D digital libraries, including:

• acquiring 3D object data—file formats, "resolution" (in terms of numbers of points or polygons) that is needed to describe the object, requirements for topological integrity or completeness of the scan. etc.
• describing object data—the standards, semantic linkages, etc., to describe the contextual and spatial data to permit the identification of individual objects and searching across data collections
• storing data (file formats, etc.)
• representing the data—conventions for modeling and displaying the object data
• interacting with the data—developing common tools and interface conventions to assist user interaction with the data and to begin building a visual literacy of 3D query and analysis

Discussion

One of the pleasant surprises during this project has been the ease of extending the modeling and analytic tools developed for one specific discipline to other research domains.The interactive growth of the tools for surface and volume modeling was another. The improvements that have resulted from the iterative process of identifying a domain research question, developing an application tool, deployment, analysis of potential applications across other research domains, and identification of new research questions have generated significant progress in developing modeling and analytic tools applicable to 3D data.

Figure 8. Bone editor developed by PRISM team with plane representing the angle of the trapezium joint surface

As 3D data-acquisition tools become more affordable and readily available, the amount of 3D data that must be described, stored, and displayed will grow dramatically. Accommodating this huge data-management challenge will require the establishment of standards and tools to analyze and add meaning to the data.

Several efforts are underway by the PRISM team or are planned to further extend the capabilities of the tools already developed and their application to domain research. In terms of infrastructure, the move from custom plug-ins to Java-based display will simplify deployment.  We are exploring alternatives to the SQL database currently used, such as object-oriented databases. Another effort to improve searching is a pilot XML search protocol developed by the National Science Foundation Biological Databases and Informatics project at ASU in conjunction with the ASU Long Term Ecological Research Metadata Committee and the Knowledge Network for Biocomplexity project at the National Center for Environmental Analysis and Synthesis. The "Xanthoria" metadata query system uses SOAP (Simple Object Access Protocol) to send XML query requests and responses and supports simultaneous Web-based querying of distributed, structurally different metadata repositories.

The analytic tools continue to develop as improvements are made in the feature extraction and region-editing applications and as more powerful techniques are developed to compare curvature, identify matches, and rank search results. Key to these efforts are the expanding partnerships with other research areas with their own unique modeling and visualization needs. Included to date are more complex anatomical data from CAT scanners and MRI, cloud-formation pattern recognition, geological erosion, and identification of targets within complex, noisy environmental data.

Figure 9. Prototype bone joint surface tool interface

Interface design continues to evolve. The project is evaluating models developed for 3D query and display by other projects, including:

• Princeton (PRINCE) 3D models search engine using Takeo Igarashi's Teddy 3D sketch interface
• National Center for Biotechnology Information Cn3D genetic viewer

The development of realistic 3D interface models that permit the researcher to sculpt the query image in 3D space is progressing, as are additional analytic tools such as planar overlays to visualize and objectively compare joint surfaces of bones. Techniques to bookmark searches to permit replication and simplify the comparison of objects in the databases are also being explored. A complex variation of bookmarks involves researchers' using the region editor and additional analytic tools such as the planar overlay to interact with the data and create their own interpretive models. Creating storage techniques for these derived, researcher-defined or modeled data, and managing "version control" to permit replication and deconstruction of the analysis is another challenge.

User evaluation of the current interface layout, color palette, and design continues using both surface and volume model data. In addition to initially developing specific bone or ceramic vessel interfaces for the different research domains, the project is working to identify commonalities and conventions to develop a unified interface model. This common design appears to be possible in initial-query interface screens, where a differentiation of interface display occurs as objects are identified, search results are returned, and researchers drill down into object data that may vary across disciplines.

Development of the current model has been an enlightening exercise in interdisciplinary project development. The translation of ideas, approaches, and vocabulary among disciplines has taken significant time and effort. Even when common vocabulary is used, the discipline-specific definitions and nuances can vary significantly.

The tools developed by the PRISM team and other researchers working to model and visualize 3D data have great potential to extend research in many disciplines. The initial challenges have focused on data acquisition and the development and display of models. Initial efforts to display images of surface models using QuickTime and plug-ins have significantly expanded research and science education as complex natural objects become approachable through such visualization. The addition of modeling and analytic tools based on surface and volume that permit objective quantification and analysis of 3D data have the potential to further extend discipline research.

As 3D data and the tools for visualization and analysis become more available, there is an increasing need for intuitive interfaces to provide gateways to the data. Digital libraries of 3D data will need to design effective processes to provide access to content created for specific projects and accommodate 3D data that they obtain from the increasing number of applications in business and industry (e.g., e-commerce, GIS, medical imaging, GPR, satellite and aerial scanning). Standards are needed for data description, storage, interchange, and searching. Conventions for display and organizing research tools are essential to generate broad acceptance and foster effective use.

Because researchers and patrons bring different strategies and approaches to their quests for information, organization and interfaces need to accommodate differences in learning styles, visual literacy, and sophistication. Evaluation data and continued research into learning styles, communication preferences, and visual communication and display are needed to guide interface design. Clearly, the development of simple, elegant, easy-to-use interfaces to accommodate the range of tools and user preferences will be a significant challenge now and in the future.

This work was supported in part by the National Science Foundation (grant IIS-9980166) and funding from the Vice Provost for Research and Economic Development at Arizona State University. The authors would like to thank all of the collaborators that make up the Partnership for Research in Spatial Modeling (PRISM) team, particularly Anshuman Razdan, Gerald Farin, Daniel Collins, Peter McCartney, Matthew Tocheri, Mary Zhu, Mark Henderson, Arleyn Simon, Mary Marzke, and David Capco.

Publishing Information

RLG DigiNews (ISSN 1093-5371) is a newsletter conceived by the members of the Research Libraries Group's PRESERV community. Funded in part by the Council on Library and Information Resources (CLIR) 1998-2000, it is available internationally via the RLG PRESERV Web site. It will be published six times in 2002. Materials contained in RLG DigiNews are subject to copyright and other proprietary rights. Permission is hereby given for the material in RLG DigiNews to be used for research purposes or private study. RLG asks that you observe the following conditions: Please cite the individual author and RLG DigiNews (please cite URL of the article) when using the material; please contact Jennifer Hartzell, RLG Corporate Communications, when citing RLG DigiNews.

Any use other than for research or private study of these materials requires prior written authorization from RLG, Inc. and/or the author of the article.

RLG DigiNews is produced for the Research Libraries Group, Inc. (RLG) by the staff of the Department of Preservation and Conservation, Cornell University Library. Co-Editors, Anne R. Kenney and Nancy Y. McGovern; Production Editors, Martha Crowe and Barbara Berger Eden; Associate Editor, Robin Dale (RLG); Technical Researchers, Richard Entlich and Peter Botticelli; Technical Coordinator, Carla DeMello; Technical Assistant, Kimberly Gazzo.

All links in this issue were confirmed accurate as of October 10, 2002.

Please send your comments and questions to RLG DigiNews Editorial Staff.