[DDI-SRG] [Fwd: FW: StatsProgs2DDI]

[Pascal Heus]

All:
J received the email below from a group working on a DDI Perl parser. Could the TIC or UOG follow-up on this?
thanks
Pascal

-----Original Message-----
From: Sandfort, Michael [mailto:Michael.Sandfort at usdoj.gov] 
Sent: Thursday, December 18, 2008 9:29 AM
To: j.b.gager at gmail.com
Subject: StatsProgs2DDI

Hello,

Your email address was listed as a contact for the StatsProgs2DDI working
group. I am an economist with the Antitrust Division at the US Dept. of
Justice. I have been working to develop a data library for some of the data
we routinely use during investigations and came upon DDI during the course
of that work. Before discovering DDI, I had developed something similar, but
very minimal in scope, which accommodated my limited metadata-preservation
needs. The programs included an XML schema for a "data dictionary" object
and a perl script for reading metadata in that schema and generating a
Postgres/PostGIS database from the ASCII data and data dictionary.

Once I discovered DDI, I thought I would drop in the DDI schema in place of
my own and rewrite the perl parser to read information from DDI-compliant
dictionaries. But the new DDI 3.0 spec is *extremely* complicated. I am
having difficulty figuring out what a data dictionary would look like which
is both minimally DDI 3.0 compliant and still would meet my limited needs.
The closest thing I have found is Aalap Doshi's interesting web-based tool,
which unfortunately does not currently accommodate the information most
critical to me -- markup describing variable names, locations in a physical
file (position and length for fixed width data and ordinal position in
delimited data), and data type
(alpha/int/float/double-precision/date/point/line/polygon). I'm also looking
at Guido Gay's tool for extracting SPSS metadata into R; but since I don't
know SPSS at all, the tool has been less useful than I had hoped. 

I'm attaching the my schema, parser and a sample dictionary as examples of
what I mean by a "simple" metadata management system. If you could point me
in the direction of any other information which might help me make the
schema DDI 3.0 compliant, I would be most appreciative! Of course, if I've
sent this message to the wrong contact, I would also appreciate any help you
can provide in routing it to a more appropriate recipient.

Thanks,
Mike Sandfort

 <<data_dictionary.xsd>>  <<xml2psql.pl>>  <<sod_dict.xml>> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: data_dictionary.xsd
Type: application/octet-stream
Size: 3576 bytes
Desc: not available
Url : http://www.icpsr.umich.edu/pipermail/ddi-srg/attachments/20081218/fd8df5d8/attachment-0002.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xml2psql.pl
Type: application/octet-stream
Size: 6893 bytes
Desc: not available
Url : http://www.icpsr.umich.edu/pipermail/ddi-srg/attachments/20081218/fd8df5d8/attachment-0003.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sod_dict.xml
Type: text/xml
Size: 36236 bytes
Desc: not available
Url : http://www.icpsr.umich.edu/pipermail/ddi-srg/attachments/20081218/fd8df5d8/attachment-0001.xml