Documentation and Metadata

According to OAIS, each digital object preserved must be "independently understandable -- a characteristic of information that has sufficient documentation to allow the information to be understood and used by the Designated Community without having to resort to special resources not widely available, including named individuals."

The metadata standard most relevant in the social science archiving area is the Data Documentation Initiative (DDI). Begun in 1995, the DDI project is an international effort to establish a standard for the content and structure of social science metadata.

Data Documentation Initiative (DDI)

DDI facilitates the creation of a comprehensive standardized set of elements to thoroughly describe datasets. This metadata can now be created in a uniform, highly structured format that is easily and precisely searchable on the Web. Thorough metadata gives a data analyst a complete view of the collection.

DDI facilitates the creation of a comprehensive set of elements to thoroughly describe datasets. This metadata can now be created in a uniform, highly structured format that is easily and precisely searchable on the Web. Thorough metadata gives a data analyst a complete view of the collection.

Many see the DDI as offering not only data users but also data producers and data archivists, a new power and flexibility to do their work and to do it effectively and efficiently.

The DDI specification is written in XML, which permits the markup, or tagging, of metadata for retrieval and repurposing across the data life cycle. The data producer imports the XML Schema into authoring software and enters text for specific project elements and attributes. The resulting file is a DDI document that can describe the dataset comprehensively in a standard language.

For more information, see the Data Documentation Initiative Web site .

Other Metadata Standards

There are several other metadata standards that can sometimes be used in the social science data context. See the Wikipedia metadata standards list for more information. Another standard increasingly used to document time series data is SDMX - Statistical Data and Metadata eXchange .

ICPSR requests thorough documentation when depositors submit their digital data for archiving. The ICPSR Data Deposit Form has several fields that map to important documentation elements. For a more in-depth discussion of the key components in documentation, see Best Practice in Creating Technical Documentation in Chapter 3 of the Guide to Social Science Data Preparation and Archiving (PDF 2M).

ICPSR is a leader of the Data Documentation Initiative (DDI) and has made a commitment to use this emerging standard in its work. There are several ways that ICPSR makes use of DDI:

• Metadata records. The metadata records that form the basis of the ICPSR online catalog are compliant with DDI. These records can be downloaded by ICPSR members and have also been converted to MARC format.

• Variable-level documentation. ICPSR has an internal data processing tool that transforms SPSS system files to other formats and to variable-level DDI. These variable-level files form the basis for ICPSR's Social Science Variables Database (SSVD).

• Technical documentation in PDF. The variable-level files described above are also the foundation for the technical documentation or codebooks that ICPSR delivers with each study.