[DDI-ADG] Resending message with reports

[Ilona Einowski]

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: DDI-GEO1 from Atle.doc
Type: application/msword
Size: 79360 bytes
Desc: not available
Url : http://lion.icpsr.umich.edu/pipermail/ddi-adg/attachments/20040301/2acdf57d/DDI-GEO1fromAtle-0001.doc
-------------- next part --------------

Relevance of SDMX to the DDI Expert Committee:         Fredric Gey 1/26/2004

BACKGROUND:
During the first meeting of the DDI Expert Committee in mid-October 2003, an XML expert (Arofan Gregory, AEON Consulting) attended from the SDMX project.   The existence of SDMX was unknown by most members of the Expert Committee, so I volunteered to investigate its relevance to the DDI.

What is SDMX?
      SDMX stands for Statistical Data and Metadata Exchange.  It is an initiative of statistical agencies of six organizations: BIS (Bank for International Settlements), ECB (European Central Bank), EuroStat (European Commission Statistical Agency), IMF (International Monetary Fund) and UN (United Nations).  Its goal is to create standards for exchange of data (and accompanying metadata) between these agencies.   >From a DDI perspective, its two most relevant projects are the:
   
   - Batch time series data exchange 
   - Metadata common vocabulary

Both of these projects seem to have been underway for almost two years and have developed extensive draft documents of process and terminology for statistical data exchange (see http://www.sdmx.org/General/Projects/GesmesTS_rel3.pdf and http://www.sdmx.org/General/Projects/MCV-draft-20031001.pdf , 192 page and 111 page documents.  However, in addition the initiative has a project which has developed a detailed information model to support the data exchange efforts, something which DDI experts agree needs to be added to the DDI. 

FOCUS OF SDMX:
The focus of the agencies is primarily the exchange of economic data that is time-sequenced (i.e time series data).  As such, most (but not all) of the data is aggregated from primary sources (i.e. it is not microdata).  Thus SDMX is most applicable to the aggregated n-cube specification of the DDI, although certain of their detailed attributes (unit of measure, collection details of when collected and whether averaged over a period) may be applicable to microdata documentation as well.

The Batch time series data exchange document presents concepts which do not seem to have been dealt with in the DDI. Among them are frequency (i.e. is the data collected daily, monthly, annually, etc), unit of measure (i.e. currency unit -- different but related to the DDI Analysis Unit), unit multiplier (i.e. is the actual data recorded in thousands, millions, etc).  To quote from the first document above:


"In general, some statistical concepts are necessary across all key families to qualify the contained information. These are:
· Reference area
· Frequency (always a dimension)
· Descriptive title (see also comment below)
· Collection (e.g., end of period, averaged, or summed over period)
· Unit (e.g., currency of denomination)
· Unit multiplier (e.g., expressed in millions)
· Availability (which institutions can a series become available to)
· Decimals (i.e., number of decimal digits used in a time series)
· Observation status (e.g., estimate, provisional, normal)
Therefore, those concepts that are not dimensions within a key family have to be present in that key family as mandatory attributes."


RELEVANCE OF SDMX:
The SDMX initiative has direct relevance to the DDI because data may become available to Data Archives from these agencies in the near future.  We have a chance to influence the development of SDMX by commenting on draft documents and by incorporating into future versions of the DDI those SDMX concepts and definitions which extend its usefulness for archiving and data exchange.  In addition SDMX has developed some prototype information models which DDI should be able to utilize in its own model development