feature request: S3, dlt and DataHub

[feriksen-personal]

Love the work you put into this. Very inspirational for my own little experiments as well.

Imho just a few parts missing to make the stack complete/perfect:

• Minio S3 storage for original files (a common source)
• ingested via dlt
• soda core for DQ/observability between the layers
• cube.dev or atscale semantic layer
• and DataHub data catalog on top.

see also:

• How to Create, Deploy & Connect Your First Model (YouTube)
• AtScale & Apache Superset Integration
• dbt-soda-core-example (github)
• datahub: dbt ingestion

[cnstlungu]

Thanks @feriksen-personal I know it's been 9 months since you opened the issue 🥲 Interesting suggestion! At the moment I'm not looking to expand the project past its current scope (limited time), but will keep it in mind for future experiments! Thanks