Home

Welcome to the Versaile Data Kit wiki!

Versatile Data Kit (VDK) is a data framework that enables Data Engineers to

🧑‍💻 develop,
▶️ run,
📊 and manage data workloads, aka data jobs

What Problem Does Versatile Data Kit Solve?

• Ingest data from different sources.
• Use Python/SQL and VDK templates to transform data.
• Package, version, and deploy data applications while dealing with credentials, retries, and reconnects.
• Provide built-in monitoring and smart notification capabilities.
• Track code and data modifications for quicker troubleshooting and version rollback.

➡️ Introduction

• Getting Started
• Installation
• First Data Job
• Examples
• Dictionary

➡️User Guide

• Overview
• VDK Components
• Installation
• Data Job Types
• Data Job Structure
• Python steps (.py)
• Ingesting data
• Data Job Properties
• Data Job Configuration
• Python dependencies (requirements.txt)
• Data Job Execution
• Resiliency
• Data Jobs Development Workflow

➡️ Community

• Community Meetings
• All Resources
• Introduction to Versatile Data Kit playlist
• Community meeting playlist
• Building ELT playlist
• Operations UI playlist