Skip to content

Create Bronze-to-Silver Pipeline Template for Snowflake #1221

@terzioglub

Description

@terzioglub

Create a bronze-to-silver pipeline template using Snowflake as the destination. This template will use either frankfurter or chess as the data source (since they don't require credentials) and showcase a complete ingestr -> transformation workflow with Snowflake. This template should be able to initialized with bruin init template-name
more about bruin init on : https://getbruin.com/docs/bruin/commands/init.html

Template Structure

Bronze Layer (Ingestr Asset):

  • Raw data ingestion using ingestr from frankfurter or chess API
  • No transformations, just data extraction to Snowflake

Silver Layer (SQL Asset):

  • Aggregates and cleans bronze data from Snowflake tables
  • Demonstrates typical transformation patterns

Implementation

Create templates/bronze-silver-snowflake/ with:

  • Bronze asset: assets/bronze_raw_data.asset.yml (ingestr from frankfurter/chess)
  • Silver asset: assets/silver_aggregated.sql (Snowflake SQL transformation)

Acceptance Criteria

  • Template created in templates/bronze-silver-snowflake/
  • Bronze layer uses ingestr asset to extract data from frankfurter or chess (no credentials needed)
  • Silver layer aggregates bronze data with Snowflake SQL transformations
  • README with Snowflake setup instructions
  • Works with bruin init bronze-silver-snowflake
  • bruin validate passes successfully
  • Assets have meaningful quality checks (column checks, custom checks)
  • bruin run executes the full pipeline successfully

Reference Templates

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions