This is a tracking issue. It can be closed when all sub-tasks are completed or obsoleted. Links to subtasks will be inserted here and tracked in the kanban.
A Source data product POC shall show:
Connect to a data supplier upstream ideally via an API
Download the data into the staging area of the OS-C Data Mesh
Ingest metadata
Monitor the source for changes/updates and (automatically) ingest new versions
Maintain history of changes
Quality control the source data through data quality checks
Facility to report deficiencies back upstream
Ability to correct deficiencies in OS-C Data Mesh in a transparent manner, keeping the original data
Update data catalog
This is a tracking issue. It can be closed when all sub-tasks are completed or obsoleted. Links to subtasks will be inserted here and tracked in the kanban.
A Source data product POC shall show:
Connect to a data supplier upstream ideally via an API
Download the data into the staging area of the OS-C Data Mesh
Ingest metadata
Monitor the source for changes/updates and (automatically) ingest new versions
Maintain history of changes
Quality control the source data through data quality checks
Facility to report deficiencies back upstream
Ability to correct deficiencies in OS-C Data Mesh in a transparent manner, keeping the original data
Update data catalog