[DDI-SRG] URN Resolution Service at Expert Committee Meeting

Wed May 7 14:44:23 EDT 2008

All:
Regarding availability and scalability, I recently looked at Amazon Web 
Services which is a misnomer as it actually is a collection of services 
to operate virtual servers, databases and storage space on Amazon web 
architecture (see http://www.amazon.com/gp/browse.html?node=3435361).

Quite a long story (we can further disuss this at the CC) but it is a 
fully scalable at any time (you're charge for the virtual appliances you 
run). A single small server instance run at I about $700/year plus minor 
costs for transactions and storage. What I like about it is that it is 
much less expensive than hosting your own server and Amazon basically is 
in charge on making sure it's available all the time, backed up, etc.. 
There are many other features (like you can duplicate machines, cover 
different geographical zones, etc.). Worth a read if you have an hour to 
spare. This seem to me as a good approach for such business critical 
services. I initially investigated this for ODaF as we're thinking about 
running a service for registration and maintenance of agency ID's.

best
*P

A. Gregory wrote:
> Achim:
>
> I should correct you: SDMX is used by banks, but also by people doing all
> other types of aggregate statistics (health, education, population, labour,
> etc.) so I think the requirement overlap is strong.
>
> Let me ask you a question:
>
> Once I have my OASIS XML Catalog, how do I maintain it?
>
> I am pretty well agreeing with you - we should start as simply as possible.
> I just wanted to point out that without a coherent plan for maintaining the
> operation of such a service, the catalog will become filled with garbage and
> then become useless (which has happened to many similar services in the
> past).
>
> I am familiar with Resolver and the approach you are suggesting. An
> alternative - which might provide a more scalable solution moving forward -
> would be to use existing SDMX registry tools to provide the same service,
> which could then be expended to include more powerful indexing and
> classification features should these prove desireable.
>
> The advantage of this approach is that we already have built mechanisms for
> performing the minimum needed maintenance (that is, registration and
> de-facto contracts regarding resource availability) which might be faster
> then deploying Resolver in a useable form. 
>
> I agree very much that we should present this idea - as functionality - to
> the Expert Committee, for exactly the reasons you propose. I think we should
> discuss implementation possibilities as a separate discussion.
>
>  Further, I would suggest that someone needs to provide a service for
> registering known DDI 3.0 using organizations, and managing the use of coded
> representations for them. We've discussed this in the past. The URNs have a
> dependency on this service, too!
>
> Cheers,
>
> Arofan
>
> -----Original Message-----
> From: Joachim Wackerow [mailto:joachim.wackerow at gesis.org] 
> Sent: Wednesday, May 07, 2008 10:27 AM
> To: A. Gregory; DDI Technical Implementation Committee
> Subject: Re: [DDI-SRG] URN Resolution Service at Expert Committee Meeting
>
> Arofan,
>
> I copy to TIC, so others can follow.
>
> I see DDI URN resolution as a crucial part of distributed DDI 
> applications. Without that nothing will be possible.
>
> Ideally URN resolution should be started with a simple XML OASIS catalog 
> file (URN to URL) going to a full blown distributed resolution service 
> like DNS. In other words a solution should be scalable, and it can be 
> started in a smaller scale.
>
> Local and remote XML catalogs can be used in a way that one is 
> delegating some resolution to other catalogs.
>
> Norman Walsh's resolver Java class is a very good start to begin with. 
> There is the stable in the Apache distribution (for a long time 
> available), and now there is a new version available with a caching 
> feature. I think this is pretty much what we need in the current state.
>
> This caching feature can be used locally in an application, but it could 
> also be used by a URN resolution service which can be remotely accessed.
>
> I think it is really important that the DDI Alliance is willing to 
> provide some kind of URN resolution service, even it is in beta state. I 
> see three benefits:
> - the service can be used by institutions, which are not able to set up 
> a similar service
> - the chosen approach can be used elsewhere by another institution
> - it is a sign to the community, that the base of distributed DDI 
> applications is started
>
> The DDI Alliance should decide to provide some kind of DDI URN 
> resolution service. It is not necessary in this state to head for a full 
> blown scenario like you described below or like the structure of DNS. As 
> I mentioned above I think in the first place a simple server with the 
> caching feature of Norman's new resolver would be a sufficient start.
>
> But the important thing is that then the DDI Alliance discussed and 
> decided this already. It would be not only a decision that some program 
> is written, but that the service is run by the DDI Alliance.
>
> BTW just a thought:
> SDMX is used mainly by banks, right? I think in social science with the 
> current state of DDI 3.0 we don't need a resolution service which has 
> the stability of a service which banks require.
>
> Other thoughts?
>
> Achim
>
> Links:
>
> Norman's article on the "old" resolver:
> http://xml.apache.org/commons/components/resolver/resolver-article.html
>
> "old" resolver:
> http://xml.apache.org/commons/components/resolver/
>
> Norman's description of the new resolver:
> http://norman.walsh.name/2007/02/06/xmlresolver
> http://norman.walsh.name/2007/02/09/resolvers
> http://norman.walsh.name/2007/02/14/resolvers
>
> new resolver:
> https://xmlresolver.dev.java.net/
>
> XML Catalogs
> http://www.oasis-open.org/committees/download.php/14809/xml-catalogs.html
>
> OASIS Entity Resolution TC
> http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=entity
>
>
>
> A. Gregory wrote:
>   
>> Achim:
>>
>> I am a bit confused. What exactly do we see this service doing?
>>
>> Is it a Java component which is part of the Foundation Tools? If so, I
>>     
> think
>   
>> it is already part of our planning.
>>
>> Is it a hosted service, to resolve URNs? If so, then it is more than a
>> simple matter of hosting the component: it is in essence the creation and
>> operation of a registry, for which our planned component will be only a
>> small part. (As you can understand, it is not simply resolving a URN, but
>> collecting and storing the locations of the metadata referred to by the
>> URNs, which involves registration as well as querying capabilities, and
>>     
> has
>   
>> to be managed and maintained over time. For example, information about how
>> long a particular resource will reside at a particular URL needs to be
>> handled - in essence a contract between a registree and the registry
>> operator/owner. These are all familiar issues from SDMX, but they are not
>> things we have dicussed in detail within DDI up to now.) And then we have
>> service-level agreements between those populating the registry, and those
>> who maintain it, to guarantee that it is not full of garbage and to avoid
>> legal liability.
>>
>> This is a fairly serious piece of software to implement and run (as we
>>     
> know
>   
>> from experience) and although I like the idea of doing it, it is an
>> operational task which may be difficult to arrange without sufficient
>> resources.
>>
>> I don't want to make too big a deal out of it, but I just want to make
>>     
> sure
>   
>> we are clear on the two very different types of tasks.
>>
>> Cheers,
>>
>> Arofan
>>
>> -----Original Message-----
>> From: ddi-srg-bounces at icpsr.umich.edu
>> [mailto:ddi-srg-bounces at icpsr.umich.edu] On Behalf Of Joachim Wackerow
>> Sent: Wednesday, May 07, 2008 8:21 AM
>> To: DDI Technical Implementation Committee
>> Subject: [DDI-SRG] URN Resolution Service at Expert Committee Meeting
>>
>> The URN resolution service should be discussed at the expert committee 
>> meeting.
>>
>> We discussed earlier the idea that the DDI Alliance can provide a DDI 
>> URN resolution service for institutions which are not able to set up on 
>> own service.
>>
>> This issue should be discussed and formally decided at the meeting.
>>
>> I mentioned this already to Mary. She thinks it can go to the tools topic.
>>
>> Pascal:
>> You should reserve some time for the URN resolution service at the tools 
>> topic. Do you have already ideas how the service can set up and which 
>> resources would be necessary, that the DDI Alliance is able to do this?
>>
>> Achim
>> _______________________________________________
>> DDI-SRG mailing list
>> DDI-SRG at icpsr.umich.edu
>> http://www.icpsr.umich.edu/mailman/listinfo/ddi-srg
>>
>>     
>
>
>
> _______________________________________________
> DDI-SRG mailing list
> DDI-SRG at icpsr.umich.edu
> http://www.icpsr.umich.edu/mailman/listinfo/ddi-srg
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.icpsr.umich.edu/pipermail/ddi-srg/attachments/20080507/8da8a330/attachment-0001.html 