This fork was created for an Oracle specific environment, where Turbovault4dbt and Datavault4dbt were used as code generators for tables and views. Load operations were carried out in a third party ETL tool, where simple 1:1 mappings between Views and tables were created. Please see this blog post for details. Besides the Oracle compatibility it offers advanced features for Oracle such as
- Link Effectivity Sattellites
- Load Views for 1:1-Mappings in ETL-Tools such as Oracle Data Integrator
- Read Views for Version 1 Satellites
- Primary Keys
- Foreign Keys
- Table Comments
- Column Comments
- Index Tablespace Modification
- Partitioning
- Helpful SQL generators to create, select, drop, delete, grant models
This fork is meant to be used together with this datavault4dbt fork. If you use Windows, use dv-starter to install and preconfigure Turbovault4dbt and Datavault4dbt easily.
TurboVault4dbt is an open-source tool that automatically generates dbt models according to our datavault4dbt-templates. It uses a metadata input of Your Data Vault 2.0 from one of the supported databases and creates ready-to-process dbt-models.
TurboVault4dbt requires a metadata analysis done by hand and stored in supported metadata storage. Furthermore, Python must be installed as TurboVault4dbt is a software written in Python.
To use the generated models, a dbt project is required. Additionally, our dbt package datavault4dbt must be used, because the dbt models are calling macros of this package.
You can find DDL scripts and templates for the metadata tables and the Excel sheet here]((https://github.com/ScalefreeCOM/turbovault4dbt/tree/main/metadata_ddl)).
Your metadata needs to be stored in the following eight tables/worksheets:
- Source Data
- Hubs
- Standard Links
- Non-Historized Links
- Standard Satellites
- Non-Historized Satellites
- Multi-Active Satellites
- Point-In-Time Tables
- Reference Data
Currently, TurboVault4dbt supports metadata input from
- Snowflake
- BigQuery
- Google Sheets
- Excel
Our developers are constantly working on adding new connectors for more databases.
To install Turbovault4dbt, follow the instructions on this page.
You can configure the connection to Your metadata storage in the config.ini. Further explanation for the configuration input can be found here.
To execute TurboVault4dbt, You need Python installed. Execute the script according to Your database, where Your metadata is stored e.g. Snowflake --> turbovault_snowflake.py, BigQuery --> turbovault_bigquery.py, and so on.
Then, a GUI will open that looks like this:
On the left side, you can select which object types you want to generate. These are:
- Stage
- Hub
- Satellite (both version 0 and version 1)
- Link
- Multiactive Satellite (both version 0 and version 1)
- Non_Historized_Satellite
- Non_Historized_Link
- Dependent_Child_Link
- Point_In_Time
- Reference Data
- Link Effectivity Satellites
- Loading Views
The right side lists all available source objects inside the connected metadata storage. You can select as many of them as you like.
You can also generate the sources.yml, properties.yml or create a Data Vault Model Visualization by ticking each respective checkbox.
Now you can click on "start" and Turbovault4dbt will generate all necessary dbt models that work with datavault4dbt!
v1.1.2 (23.01.2024) - Current Version
v1.1.1 (24.05.2023)
v1.1.0 (22.05.2023)
v1.0.3 (16.02.2023)
v1.0.2 (13.02.2023)
v1.0.1 (30.01.2023)
v1.0.0 (26.01.2023)
