[Task/Feature/Issue] NG Catalog addition: Max Planck Society?

[jolson-ibm]

Is there an existing issue for this?

• I have searched the existing issues

Task/Feature/Issue Details

Determine the feasibility of adding any data sets the Max Planck Society is producing.

This will be the second data source added to the catalog besides HF.

To Dos:

1. Figure out where their data sets are hosted.
2. Figure out their data set documentation model (if any...)
3. Figure out their data set access options (APIs, REST, etc)

Once the above is done we can assess if we can fit it into the current NG catalog model, or how we need to amend it.

Additional Context

``

[jolson-ibm]

Introducing EDMOND - the Open Research Data Repository of the Max Planck Society

1,235 data sets
Free form basic search. Faceted search. Advance search.
Search appears to be performed via parameterization of the URL

"Data card" consists of a description, various categories of tags, related publication(s), citations, a license, metadata, "terms", and versioning.

It also contains a "contact owner" link.

Does not look like there is an API here, so scraping will probably be on the table.

I'll do some more digging.

[jolson-ibm]

ALSO...

There is a potential integration point with OTDI with Bloxberg, the blockchain-for-science solution Max Planck is contributing to.

The blockchain offers two value props:

1. No one controls it - it is fully decentralized
2. When you write information there, it is tamper proof and there forever.

No other storage technology offers up that exact value proposition. Of course, these come with a cost....

For OTDI: we are supposed to be scoring dataset documentation against specifications.
We simply write a hash of the data set documentation, a hash of the dataset itself, a hash of the version of the spec, a hash of the version of the scoring engine and the score itself, and it is preserved forever, tamper proof, no one "controls" the information, and everyone can access it...Any other way of storing this information would have to be controlled by someone, and therefore can be tampered with in some way - the org could collapse and the info disappear, they could cut off access, they could maliciously change it, etc...