[DDI-ADG] Concerns about nCube spec?

[Wendy Thomas]

Hi,

I could be wrong but I believe this is addressing the issue that aggregate
data is freqently held in 2 and 3 dimensional storage systems
(spreadsheets, layered spreadsheets) or bundled as data objects where a
"cell" contains an array of items in a fixed order. The locMap does not
address any new types of storage and DDI only address fixed and delimited
records. This is really a separate issue from aggregate data description
as any type of file could be stored in these alternative formats. So what
the locMap supplies is the link to the data item (cell) description by
giving you its matrix (nCube) number and cell coordinates. Currently the
phyLoc line can only provide a pointer to a fixed format or delimited
file. This is one of the problems for the "European aggregate" data. CBS
was using CUBE storage (3 dimensional) and I know that Jostein had
described their data storage system as one of those using bundled arrays.

NHGIS is making use of the current aggregate description to search for
data items and tables, create the table template on the fly and populate
it with data from a fixed format data file containing multiple nCubes per
record of data. NESSTAR uses it for describing and manipulating files of
single nCubes for multiple locations.

Wendy Thomas

On Wed, 17 Mar 2004, Julie Linden wrote:

> Hello everyone,
>
> I've been thinking about how to address the "aggregate" part of our
> Working Group's charge. The "Possible Configuration of DDI Working Groups"
> document that was distributed at the Expert Committee meeting in October
> states: "While considerable time and effort have already gone into the
> creation of an aggregate/tabular extension to the existing DDI
> specification (nCubes), there is concern that the aggregate model may be
> overly complex. The group needs to take a fresh look at this issue."
>
> As someone who is just beginning to get familiar with how the current DDI
> handles aggregate data, it's hard for me to begin envisioning how it could
> be simplified or overhauled. I thought that perhaps a starting point would
> be to review what concerns have been raised. I read through the Structural
> Reform Group's postings on ezboard, and found one comment that suggests a
> concern, but doesn't spell it out:
>
> "Logical  Physical file format mapping: How are the logical concepts in
> the DDI mapped to the underlying physical files? What kinds of physical
> file formats are there (rectangular, cards, SPSS, STATA, SAS, Census
> aggregate data, European aggregate data)? Should DDI even be tackling this
> question? There is an existing difference of opinion already regarding
> this in the nCubes specification."
>
> Can someone on this group describe the issues/concerns explicitly?
>
> thanks,
> Julie
>
>
>
>
> _______________________________________________
> DDI-ADG mailing list
> DDI-ADG at icpsr.umich.edu
> http://www.icpsr.umich.edu/mailman/listinfo/ddi-adg
>

Wendy L. Thomas                          Phone: +1 612.624.4389
Data Access Core Director		 Fax:   +1 612.626.8375
Minnesota Population Center              Email: wlt at pop.umn.edu
University of Minnesota
537 Heller Hall
271 19th Avenue South
Minneapolis, MN 55455