Intake Data Catalog For LOCA2 Datasets

[BenGalewsky]

As a user of the Illinois CLIMAS Analysis Facility I want an accurate data catalog so I can find the data I need to complete my analysis

Problem

The LOCA2 dataset, downloaded from UCSD servers sits in 27 different models. Each model has four scenarios and each scenario has several member IDs. Assembling a coherent analysis from these zarr stores spread across multiple directories is challenging.

An Intake Catalog (https://intake.readthedocs.io/en/latest/user2.html) makes it much easier to find and collect these stores for an analysis.

Approach

Define a catalog that can represent the LOCA2 dataset and allows for searches that meet some of the most common use cases. Particular priority should be given to queries needed for the Illinois Department of Health science gateway.

If possible, the catalog should dynamically created as part of the LOCA2 Dagster assets.

[mailesasaki]

Hello, I have code that uses the Intake Catalog here: https://github.com/mailesasaki/climate_map/blob/main/LOCA2/LOCA2_processor.py

The catalog is in Keeling with the LOCA2 dataset I have. Should I send it over as well?

[BenGalewsky]

The catalog is in Keeling with the LOCA2 dataset I have. Should I send it over as well?

Yes - I don't have access to Keeling. How big is the yaml file? Can you check it in, or you could just put it in box or a google drive

[mailesasaki]

It's actually a csv file I loaded using intake-esm (https://intake-esm.readthedocs.io/en/stable/). It's pretty small so I think I can upload it in this comment.

LOCA2_catalog.csv