-
Notifications
You must be signed in to change notification settings - Fork 52
Labels
good first issueGood for newcomersGood for newcomers
Description
Create a bronze-to-silver pipeline template using Snowflake as the destination. This template will use either frankfurter or chess as the data source (since they don't require credentials) and showcase a complete ingestr -> transformation workflow with Snowflake. This template should be able to initialized with bruin init template-name
more about bruin init on : https://getbruin.com/docs/bruin/commands/init.html
Template Structure
Bronze Layer (Ingestr Asset):
- Raw data ingestion using ingestr from frankfurter or chess API
- No transformations, just data extraction to Snowflake
Silver Layer (SQL Asset):
- Aggregates and cleans bronze data from Snowflake tables
- Demonstrates typical transformation patterns
Implementation
Create templates/bronze-silver-snowflake/ with:
- Bronze asset:
assets/bronze_raw_data.asset.yml(ingestr from frankfurter/chess) - Silver asset:
assets/silver_aggregated.sql(Snowflake SQL transformation)
Acceptance Criteria
- Template created in
templates/bronze-silver-snowflake/ - Bronze layer uses ingestr asset to extract data from frankfurter or chess (no credentials needed)
- Silver layer aggregates bronze data with Snowflake SQL transformations
- README with Snowflake setup instructions
- Works with
bruin init bronze-silver-snowflake -
bruin validatepasses successfully - Assets have meaningful quality checks (column checks, custom checks)
-
bruin runexecutes the full pipeline successfully
Reference Templates
- Existing templates: https://getbruin.com/docs/bruin/getting-started/templates.html
- Chess template: https://getbruin.com/docs/bruin/getting-started/templates-docs/chess-README.html
- Frankfurter template: https://getbruin.com/docs/bruin/getting-started/templates-docs/frankfurter-README.html
Metadata
Metadata
Assignees
Labels
good first issueGood for newcomersGood for newcomers