[DDI-ADG] progress on aggregate data?

[Katherine McNeill-Harman]

At the end of yesterday's phone conversation, I mentioned to J that I 
thought that we had the least documentation on what changes/structures to 
suggest for aggregate data (i.e. no single spreadsheet from which we were 
working).  So he's going to try to compile something but, as he said in his 
other email, we need to pool our thoughts on this.  So I'm starting the 
ball rolling.

Following are the goals for aggregate data we'd outlined in Edinburgh; what 
changes have we agreed upon that will accomplish these?
- Accommodate data files in formats w/ integrated data and metadata (e.g. 
Excel files) self documenting.
- Evaluate broad utility of nCubes
- Need ability to describe method of aggregation
- Need of additional tags to describe aggregate data (not nCubes)
- Review tag names
- Role of modules for different kinds of data
- Align to SDMX

I've been looking over my notes on our changes/proposals; here is what's 
been said in principle, but again, we'll want to document this and consider 
how to accomplish it (I put the date when I have it discussed):

- DDI is missing a way to describe the physical structure of a spreadsheet; 
need physical description for rows/columns/layers to say how they relate to 
each other; this would enable machine-actionable collapsing of rows or 
columns (e.g. collapsing of age groups) and creation of subtotals and 
totals; it should also accommodate various levels and irregular nested 
categories, and be able to identify the lowest level (8/9)
- (8/9)
- need to be able to mark up and represent existing tables (e.g. from print 
volumes) (8/16)
- enable creation of a single file containing both data and metadata; this 
format would be optional and could be applied when appropriate (8/16)
- unlike SDMX, enable a single file that contains both the data and the 
structure (8/23)
- be able to apply attribute information at all levels (from cells on up); 
could add to n-cube in the measure element a sub-element that defines 
attributes that can be attached to any level; provide a structure by which 
authors could define these.  However, it's not the case as with other 3.0 
features that items at a lower level override things at a higher level; 
therefore, the structure will need to be such that it's clear that 
attributes can be defined only at one chosen level (i.e. can't have 
conflicting attributes at different levels). (8/23)
- ability to locate the desired cell within the cube (8/23)
- hinging is important, yet may be addressed by comparative data group; SRG 
liaison will check (8/23)

I'd ask others to help add to and clarify these.  In addition, many of the 
above I just have articulated as goals, so I'm not clear if we've yet 
figured out how to accomplish these.

Kate

___________________________________________
Katherine McNeill-Harman
Data Services Librarian
Dewey Library for Management and Social Sciences
Massachusetts Institute of Technology
77 Massachusetts Avenue, E53-100
Cambridge, MA 02139
mcneillh at mit.edu
617-253-0787