[DDI-ADG] Concerns about nCube spec?

[Julie Linden]

Thanks for the response. I'm just trying to understand the issues here; I 
apologize to others for whom these issues may be very clear and familiar. 
Would one approach for our group be to try to figure out how to describe 
these other types of storage systems that the DDI does not currently cover? 
And if so, would we try to do so within the DDI framework? How would our 
efforts fit into the work that the Structural Reform Group is doing -- do 
we need to wait for the SRG to get further along in its work? If the DDI is 
going to end up as more modular -- and maybe that's not the right word or 
even the right concept, so I hope someone will correct me! -- then 
aggregate data could potentially be described by the appropriate aggregate 
data "module" -- e.g. there would be one module for fixed / delimited 
records, one module for bundled arrays, one for CUBE storage, etc?

Thanks
Julie

At 01:14 PM 3/17/2004 -0600, Wendy Thomas wrote:
>Hi,
>
>I could be wrong but I believe this is addressing the issue that aggregate
>data is freqently held in 2 and 3 dimensional storage systems
>(spreadsheets, layered spreadsheets) or bundled as data objects where a
>"cell" contains an array of items in a fixed order. The locMap does not
>address any new types of storage and DDI only address fixed and delimited
>records. This is really a separate issue from aggregate data description
>as any type of file could be stored in these alternative formats. So what
>the locMap supplies is the link to the data item (cell) description by
>giving you its matrix (nCube) number and cell coordinates. Currently the
>phyLoc line can only provide a pointer to a fixed format or delimited
>file. This is one of the problems for the "European aggregate" data. CBS
>was using CUBE storage (3 dimensional) and I know that Jostein had
>described their data storage system as one of those using bundled arrays.
>
>NHGIS is making use of the current aggregate description to search for
>data items and tables, create the table template on the fly and populate
>it with data from a fixed format data file containing multiple nCubes per
>record of data. NESSTAR uses it for describing and manipulating files of
>single nCubes for multiple locations.
>
>Wendy Thomas
>
>On Wed, 17 Mar 2004, Julie Linden wrote:
>
> > Hello everyone,
> >
> > I've been thinking about how to address the "aggregate" part of our
> > Working Group's charge. The "Possible Configuration of DDI Working Groups"
> > document that was distributed at the Expert Committee meeting in October
> > states: "While considerable time and effort have already gone into the
> > creation of an aggregate/tabular extension to the existing DDI
> > specification (nCubes), there is concern that the aggregate model may be
> > overly complex. The group needs to take a fresh look at this issue."
> >
> > As someone who is just beginning to get familiar with how the current DDI
> > handles aggregate data, it's hard for me to begin envisioning how it could
> > be simplified or overhauled. I thought that perhaps a starting point would
> > be to review what concerns have been raised. I read through the Structural
> > Reform Group's postings on ezboard, and found one comment that suggests a
> > concern, but doesn't spell it out:
> >
> > "Logical  Physical file format mapping: How are the logical concepts in
> > the DDI mapped to the underlying physical files? What kinds of physical
> > file formats are there (rectangular, cards, SPSS, STATA, SAS, Census
> > aggregate data, European aggregate data)? Should DDI even be tackling this
> > question? There is an existing difference of opinion already regarding
> > this in the nCubes specification."
> >
> > Can someone on this group describe the issues/concerns explicitly?
> >
> > thanks,
> > Julie
> >
> >
> >
> >
> > _______________________________________________
> > DDI-ADG mailing list
> > DDI-ADG at icpsr.umich.edu
> > http://www.icpsr.umich.edu/mailman/listinfo/ddi-adg
> >
>
>Wendy L. Thomas                          Phone: +1 612.624.4389
>Data Access Core Director               Fax:   +1 612.626.8375
>Minnesota Population Center              Email: wlt at pop.umn.edu
>University of Minnesota
>537 Heller Hall
>271 19th Avenue South
>Minneapolis, MN 55455