Best Practices Across the Data Life Cycle
The Best Practices provided here reflect the work of 25 individuals who came together at Schloss Dagstuhl, in Wadern Germany, in November 2008. The documents are the first attempt to define procedures for using DDI optimally and as such are not considered "final."
The best practice authors and editors welcome your feedback on the usability of the documents and priorities for new best practices. Please send your comments and suggestions to
ddi-bp-editors@icpsr.umich.edu
Series Editors (as of May 2009): Mary Vardigan, Stefan Kramer, Joachim Wackerow, Janet Eisenhauer
Introduction
Best practices in workflow operate within the wider context of the social science community's activities in information and knowledge. One way of viewing these activities is to organize them according to a life cycle model consisting of identifiable stages. Each stage is discrete and contains within it a set of related activities. How these activities are organized, managed, and carried out is in part dependent on the workflow of the organization or agency doing the work.
Modern research is conducted within a community of agencies, organizations, and individuals that together support the generation of new data and knowledge. Even though individual research investigators receive substantial portions of funds from funding agencies and councils, the large investments in science are made in research teams or programs, which often are interdisciplinary and international in composition.
The management of research resources and materials within a research community involves stewardship responsibilities by all of the stakeholders making up the community. From a life cycle perspective, each stakeholder will have primary custodial responsibility for research resources within or across stages of the life cycle. A major challenge in the overall stewardship of research data and metadata is coordinating the transfer of custodial responsibilities of the metadata and data as they pass from stage to stage in the data life cycle.
The DDI community believes that the best practices in metadata creation and use take place when the following ideals are observed:
- Metadata production should not be considered solely as an afterthought.
- Metadata should facilitate other activities in the overall workflows of the data life cycle.
- Metadata that are generated throughout the life cycle should be integrated with other metadata.
- Published metadata should be versioned and preserved to maintain transparency, but never discarded - if it were, this would obscure earlier stages of data development.
- Metadata producers must be aware of their role in the larger research picture. At each stage of the life cycle, the metadata produced/prioritized may not meet the needs of others in the life cycle. This underscores the need for better communication, awareness, and articulation of metadata across the entire data life cycle.
- Key stakeholders like funding agencies need to be educated about the metadata requirements of all stages of the data life cycle and adopt measures to ensure that good stewardship and citizenship are practiced. They become the standard bearers for metadata best practices.
DDI Best Practices
- Implementation and Governance
- This best practice covers the process of developing agreements and guidelines for implementation and governance of DDI as a metadata scheme for a community. The best practice recognizes that different communities will, by necessity, have different processes for this, and offers a checklist of issues for communities to consider before implementing DDI.
- Work flows - Data Discovery and Dissemination: User Perspective
- Describes the best practices for metadata producers to provide end users with the resources for data discovery and dissemination.
- Work flows - Archival Ingest and Metadata Enhancement
- This Best Practice discusses workflows for DDI usage in the context of archival ingest and metadata enhancement, beginning at the point of the handoff between the data provider and the archive.
- Work flows for Metadata Creation Regarding Recoding, Aggregation and Other Data Processing Activities
- This best practice discusses the capturing, in DDI metadata, of the processes of data aggregation, recoding and data processing.
- Controlled Vocabularies
- The benefits of using controlled vocabularies within DDI metadata creation are the primary underlying principles behind this Best Practice document.
Use of a controlled vocabulary enhances consistency and efficiency in the production of DDI metadata, gives precision in searching the DDI metadata, and allows semantic and technical interoperability between organisations creating DDI instances.
- Creating a DDI Profile
- This document outlines recommended best practices for creating a local DDI 3.0 Profile, which is a subset of DDI 3.0 fields to be used by an organization or shared by a community of users.
The DDI 3.0 specification is extensive and designed to cover a multiplicity of use cases. However, not all of DDI's possible applications will be relevant at the level of specific organizations or user communities. By creating and implementing user-specific profiles, organizations will:
Ensure that DDI documentation is suited to their particular requirements;
Expedite and simplify DDI production and processing;
Optimize interoperability and facilitate document sharing with other DDI users.
The guidelines included in this document are intended to assist potential users in building a technically accurate and complete profile that will serve as an effective tool in managing DDI instances as well as data sharing operations.
- DDI 3.0 Schemes
- The intention of resource packages in DDI is to provide a publishing structure that promotes reuse of common material by removing certain metadata content from the context of a specific study and publishing it for the primary purpose of reuse. Reuse of common material such as concept schemes, geographic location schemes, and coding schemes provides implicit comparison between those studies that reference a common source of information. Publication of DDI schemes as resource packages both within organizations or projects and for broader public access requires the consideration of how these materials may be used and incorporated into other studies. Resource packages need to be structurally designed to facilitate reuse among the potential users of the content. In addition, the publication of similar schemes as resource packages supports potential comparison and publication of the structured relationship between two or more schemes. Resource packages are published with the primary intention that their contents be reused and therefore the expectation that they will be maintained and available in perpetuity.
- Versioning and Publication
- One of the objectives in creating DDI 3.0 was full machine-actionability. This requires strict versioning of objects so that users understand the change history of the resources they are using. This Best Practice is designed to provide some guidelines to metadata creators and publishers about how to version metadata and publish it for others to use.
- DDI as Content for Registries
- The authors believe that it is currently premature to recommend a 'best practice' for interacting effectively with registries. Instead, this document defines registries and provides a justification for using them. It begins by giving relevant definitions and discussing the general business case and then goes on to describe from a high-level perspective the use of registries that allow researchers to access material. It concludes by describing examples of researcher use cases.
- Management of DDI 3.0 Unique Identifiers
- This best practices document looks at a possible way to design components that can be combined to create DDI applications. Given that object-oriented design is the most common programming paradigm, and that systems are often based around service-oriented principles, and given the modular design of DDI 3.0 itself, this document provides an architectural model that can be a reference point for implementers. The document also takes into consideration issues of maintenance and management of DDI applications, and discusses best practices for application documentation and configuration. The focus is on interoperability of DDI applications.
- DDI 3.0 URNs and Entity Resolution
- This document is not quite a best practice but rather a recommendation about appropriate architecture for the effective resolution of DDI URNs. The recommended architecture is based on standard and tested technologies put together in order to facilitate URN resolution needs. Along the way we describe the consequences for the various parties involved and the relationship of DDI URNs to other resolution mechanisms. This document does not deal with the latter in depth, as that would be the subject of a white paper in its own right.
- High-Level Architectural Model for DDI Applications
- This best practices document looks at a possible way to design components that can be combined to create DDI applications. Given that object-oriented design is the most common programming paradigm, and that systems are often based around service-oriented principles, and given the modular design of DDI 3.0 itself, this document provides an architectural model that can be a reference point for implementers. The document also takes into consideration issues of maintenance and management of DDI applications, and discusses best practices for application documentation and configuration. The focus is on interoperability of DDI applications.
DDI Best Practice Definitions
