Ingestion

VDK Ingestion is the process of

• extracting data from various sources (HTTP APIs, Databases, CSV, etc.).
• Ensuring data fidelity with minimal transformations.
• Loading data to your preferred destination (database, cloud storage).

Ingestion examples:

➡ Ingesting data from REST API into Database
➡ Ingesting data from DB into Database
➡ Ingesting local CSV file into Database
➡ Incremental ingestion using Job Properties
➡ Ingesting data from an authenticated REST API using Secrets

Jupyter Ingestion Tutorial

➡ VDK Ingestion Tutorial with Jupyter Notebooks

Ingestion Videos

▶️ Data Ingestion Intro
▶️ Incremental Ingestion

All VDK Examples

All VDK examples can be found here

➡️ Next section: Transformation