[DDI-SRG] [DDI-UOG] [Fwd: FW: StatsProgs2DDI]

[Katherine McNeill]

Stefan and I conferred about this; I'd be interested to hear the TIC's thoughts on this as well.

We might actually say that this is something for the DDI Users list.  At this point neither TIC for UOG I believe are responsible for/charged with helping individuals use the data (TIC I understand to focus on development of the specification and UOG on outreach and usability, but neither on individual user support).

Maybe that's a gap in service that we have (i.e. there is no working group charged with user support), but then again I think that for many open-source projects, because there's not a specific service provider that you're paying for support, the help falls on the shoulder of the "community," an ambiguous entity that in this case I think is best represented by the user's list.

Other thoughts?
Kate

-----Original Message-----
From: ddi-uog-bounces at icpsr.umich.edu [mailto:ddi-uog-bounces at icpsr.umich.edu] On Behalf Of Pascal Heus
Sent: Thursday, December 18, 2008 10:17 AM
To: DDI Structural Reform Working Group.; DDI Usability and Outreach Working Group
Subject: [DDI-UOG] [Fwd: FW: StatsProgs2DDI]

All:
J received the email below from a group working on a DDI Perl parser. Could the TIC or UOG follow-up on this?
thanks
Pascal

-----Original Message-----
From: Sandfort, Michael [mailto:Michael.Sandfort at usdoj.gov] 
Sent: Thursday, December 18, 2008 9:29 AM
To: j.b.gager at gmail.com
Subject: StatsProgs2DDI

Hello,

Your email address was listed as a contact for the StatsProgs2DDI working
group. I am an economist with the Antitrust Division at the US Dept. of
Justice. I have been working to develop a data library for some of the data
we routinely use during investigations and came upon DDI during the course
of that work. Before discovering DDI, I had developed something similar, but
very minimal in scope, which accommodated my limited metadata-preservation
needs. The programs included an XML schema for a "data dictionary" object
and a perl script for reading metadata in that schema and generating a
Postgres/PostGIS database from the ASCII data and data dictionary.

Once I discovered DDI, I thought I would drop in the DDI schema in place of
my own and rewrite the perl parser to read information from DDI-compliant
dictionaries. But the new DDI 3.0 spec is *extremely* complicated. I am
having difficulty figuring out what a data dictionary would look like which
is both minimally DDI 3.0 compliant and still would meet my limited needs.
The closest thing I have found is Aalap Doshi's interesting web-based tool,
which unfortunately does not currently accommodate the information most
critical to me -- markup describing variable names, locations in a physical
file (position and length for fixed width data and ordinal position in
delimited data), and data type
(alpha/int/float/double-precision/date/point/line/polygon). I'm also looking
at Guido Gay's tool for extracting SPSS metadata into R; but since I don't
know SPSS at all, the tool has been less useful than I had hoped. 

I'm attaching the my schema, parser and a sample dictionary as examples of
what I mean by a "simple" metadata management system. If you could point me
in the direction of any other information which might help me make the
schema DDI 3.0 compliant, I would be most appreciative! Of course, if I've
sent this message to the wrong contact, I would also appreciate any help you
can provide in routing it to a more appropriate recipient.

Thanks,
Mike Sandfort

 <<data_dictionary.xsd>>  <<xml2psql.pl>>  <<sod_dict.xml>>