Create mapping of repository to API to allow download of externally hosted datasets

[jaybee84]

Related to and supportive of: https://sagebionetworks.jira.com/browse/CPO-1700

We need to create a mapping to allow Portal config to trigger the download cart functionality and use the APIs to allow download of datasets directly through CCKP.

Orion has previously compiled a list of the top repositories where datasets reside, and their available APIs: https://docs.google.com/spreadsheets/d/1ahkfMFHVOA6w5R5jel_19W5b292E1jumEiZ9OKOZ20k/edit?usp=sharing

[aclayton555]

Effort compliments planned work in https://sagebionetworks.jira.com/browse/CPO-1564, which we would like to have done by the site visit (Sept 2025)

[Bankso]

This is far from being useful in its current state, but I was thinking about ways to download sequencing data files associated with GEO entries today and wrote up some of my observations/ideas here: https://docs.google.com/document/d/1Kc9TgowucftIjBR5Izbq4m9HCbV7TCg4SlRJo7Ro014/edit?pli=1&tab=t.g0inh8nretoi

[aclayton555]

25-7 Kick Off: Need to understand what information is needed by the portals team. there are a couple of ways to interact with these repositories to facilitate this, so we need to know what to curate (if anything). May also be complimentary to Croissant work.

User story: As a researcher interested in using datasets found via the CCKP, I want to be able to add files to my download cart, as well as metadata. I ideally want this to be as similar of an experience as possible, regardless of whether the files are hosted in Synapse vs an external repo like GEO.

This may necessitate a combination of tools: e.g. Geofetch and SRA tools.

• For either of these tools, which will allow downloading files as a package? SRA tools can do this, but maybe not Geofetch.
• For the metadata, Geofetch may be better for this.

[aclayton555]

Latest discussions tracked in: https://sagebionetworks.jira.com/browse/PORTALS-3570

TL;DR - tech team to focus research into downloading dataset collections toward Sept timeline (see https://sagebionetworks.jira.com/browse/PLFM-8959)

Work related to downloading external datasets - Jineta suggests curation research target timeline should be mid 2026 Q1, and end of 2026 Q1 for Portal implementation

[aclayton555]

External dataset download work tracked in: https://sagebionetworks.jira.com/browse/TECH-186