Skip to content

Update spec for new staging areas for linking updates and deletes #59

@aaclan-ebi

Description

@aaclan-ebi

Update spec according to what has been agreed on last post DCP2 demo meeting

Related to ebi-ait/dcp-ingest-central#695
Related slack thread https://humancellatlas.slack.com/archives/C01360XN04S/p1646223040031019

Notes:

  • New staging area enum types - normal, delta, updated (or latest tbd)
  • "updated" staging areas will always contain the latest set of metadata, links, descriptor files
  • At some point, we may want to add support for updated data files and in that case, there will be data files present in the staging area
  • Sequence diagram: https://collabuml.com/update-links

A new staging area which we will require Data Import to reset a project and import the staging area like a new project.
Pro(s): The design is simpler and more straightforward and covers all scenarios for metadata updates and deletes.
Con(s): It may be expensive for an extremely large dataset (e.g. Tabula Muris), in which case we could use the delta staging area approach once we really need to.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions