Skip to content

Commit e2eefdf

Browse files
Copilottrouze
Copilot
and
trouze
authored
feat: distributable GitHub Action for dbt Cloud YAML schema validation (#27)
* Initial plan * feat: add distributable validate GitHub Action for YAML schema validation Agent-Logs-Url: https://github.com/dbt-labs/terraform-dbtcloud-as-yaml/sessions/bdf0be0a-5268-4e3c-9985-40618cd9f2da Co-authored-by: trouze <44027587+trouze@users.noreply.github.com> * docs: add yaml-validation guide page for the validate GitHub Action Agent-Logs-Url: https://github.com/dbt-labs/terraform-dbtcloud-as-yaml/sessions/fb4dfe1a-7c77-4a09-ad63-e24b40b804c6 Co-authored-by: trouze <44027587+trouze@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: trouze <44027587+trouze@users.noreply.github.com>
1 parent 2839611 commit e2eefdf

7 files changed

Lines changed: 329 additions & 0 deletions

File tree

‎.github/workflows/validate.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# validate.yml — tests the validate GitHub Action on every PR and push to main
2+
# that touches the action definition, schema, or test fixtures.
3+
4+
name: Validate Action
5+
6+
on:
7+
push:
8+
branches: [main]
9+
paths:
10+
- "validate/**"
11+
- "schemas/**"
12+
- ".github/workflows/validate.yml"
13+
pull_request:
14+
paths:
15+
- "validate/**"
16+
- "schemas/**"
17+
- ".github/workflows/validate.yml"
18+
19+
permissions:
20+
contents: read
21+
22+
jobs:
23+
test-valid:
24+
name: Valid YAML passes
25+
runs-on: ubuntu-latest
26+
steps:
27+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
28+
- uses: ./validate
29+
with:
30+
file: validate/tests/valid.yml
31+
32+
test-invalid:
33+
name: Invalid YAML is rejected
34+
runs-on: ubuntu-latest
35+
steps:
36+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
37+
- id: validate
38+
uses: ./validate
39+
with:
40+
file: validate/tests/invalid.yml
41+
continue-on-error: true
42+
- name: Assert validation failed
43+
run: |
44+
if [ "${{ steps.validate.outcome }}" != "failure" ]; then
45+
echo "Expected validation to fail for invalid.yml, but it did not."
46+
exit 1
47+
fi
48+
echo "Validation correctly rejected invalid.yml."

‎README.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,38 @@ schedule_hours: [6, 18] # UTC
284284

285285
---
286286

287+
## Validate your YAML in CI (GitHub Action)
288+
289+
Use the bundled `validate` action to check your `dbt-config.yml` against the
290+
JSON schema **before** running Terraform or supplying any dbt Cloud credentials.
291+
This catches typos and structural errors early in your pull-request workflow.
292+
293+
```yaml
294+
# .github/workflows/dbt-cloud-validate.yml
295+
name: Validate dbt Cloud YAML
296+
297+
on: [push, pull_request]
298+
299+
jobs:
300+
validate:
301+
runs-on: ubuntu-latest
302+
steps:
303+
- uses: actions/checkout@v4
304+
- uses: dbt-labs/terraform-dbtcloud-as-yaml/validate@v1 # pin to a release tag or commit SHA
305+
with:
306+
file: dbt-config.yml # default; omit if your file is named dbt-config.yml
307+
```
308+
309+
| Input | Default | Description |
310+
|---|---|---|
311+
| `file` | `dbt-config.yml` | Path to the YAML file to validate (relative to the workspace root). |
312+
313+
The action exits with code `1` and prints a structured error report when the
314+
file does not conform to the schema, so your CI run fails fast without needing
315+
Terraform credentials.
316+
317+
---
318+
287319
## Requirements
288320

289321
- Terraform >= 1.0

‎docs/guides/yaml-validation.md‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# YAML Validation Action
2+
3+
Validate your `dbt-config.yml` against the dbt Cloud Terraform YAML schema **before** running Terraform or supplying any dbt Cloud credentials. This catches typos, missing required fields, and structural errors early in your pull-request workflow.
4+
5+
## Overview
6+
7+
The `validate` action is a composite GitHub Action shipped inside this repository. It uses [`check-jsonschema`](https://check-jsonschema.readthedocs.io/) to validate your YAML file against [`schemas/v1.json`](https://raw.githubusercontent.com/dbt-labs/terraform-dbtcloud-as-yaml/main/schemas/v1.json).
8+
9+
- **No Terraform required** — no `terraform init`, no provider downloads
10+
- **No dbt Cloud credentials required** — purely structural validation
11+
- **Fast** — typically completes in under 30 seconds
12+
- **Clear error messages** — reports every violation with a JSON path so you know exactly what to fix
13+
14+
---
15+
16+
## Quick Start
17+
18+
```yaml title=".github/workflows/validate.yml"
19+
name: Validate dbt Cloud YAML
20+
21+
on:
22+
push:
23+
pull_request:
24+
25+
jobs:
26+
validate:
27+
runs-on: ubuntu-latest
28+
steps:
29+
- uses: actions/checkout@v4
30+
- uses: dbt-labs/terraform-dbtcloud-as-yaml/validate@v1
31+
with:
32+
file: dbt-config.yml # optional — this is the default
33+
```
34+
35+
The job exits with code `1` and prints a structured error report when the file does not conform to the schema, causing your CI run to fail immediately — before any Terraform or credential steps run.
36+
37+
---
38+
39+
## Inputs
40+
41+
| Input | Required | Default | Description |
42+
|-------|----------|---------|-------------|
43+
| `file` | No | `dbt-config.yml` | Path to the dbt Cloud YAML configuration file to validate, relative to the repository root. |
44+
45+
---
46+
47+
## Example: Validate before Terraform Plan
48+
49+
Add the validation step at the top of your existing Terraform CI workflow so bad YAML is caught before Terraform even initialises:
50+
51+
```yaml title=".github/workflows/ci.yml"
52+
name: CI — Validate and Plan
53+
54+
on:
55+
pull_request:
56+
branches: [main]
57+
paths:
58+
- "dbt-config.yml"
59+
- "**.tf"
60+
61+
permissions:
62+
contents: read
63+
pull-requests: write
64+
65+
jobs:
66+
validate:
67+
name: Validate YAML
68+
runs-on: ubuntu-latest
69+
steps:
70+
- uses: actions/checkout@v4
71+
- uses: dbt-labs/terraform-dbtcloud-as-yaml/validate@v1
72+
with:
73+
file: dbt-config.yml
74+
75+
plan:
76+
name: Terraform Plan
77+
runs-on: ubuntu-latest
78+
needs: validate # only runs when YAML is valid
79+
env:
80+
TF_VAR_dbt_account_id: ${{ secrets.DBT_ACCOUNT_ID }}
81+
TF_VAR_dbt_token: ${{ secrets.DBT_TOKEN }}
82+
TF_VAR_dbt_host_url: "https://cloud.getdbt.com"
83+
TF_VAR_environment_credentials: ${{ secrets.ENVIRONMENT_CREDENTIALS }}
84+
steps:
85+
- uses: actions/checkout@v4
86+
- uses: hashicorp/setup-terraform@v3
87+
- run: terraform init
88+
- run: terraform plan -no-color
89+
```
90+
91+
---
92+
93+
## Example Error Output
94+
95+
When a YAML file fails validation the action prints a structured report and exits with code `1`:
96+
97+
```
98+
Validating 'dbt-config.yml' against dbt Cloud YAML schema v1...
99+
Schema validation errors were encountered.
100+
dbt-config.yml::$: 'version' is a required property
101+
dbt-config.yml::$.account: 'host_url' is a required property
102+
dbt-config.yml::$.projects[0].environments[0].credential: 'token_name' is a required property
103+
Error: Process completed with exit code 1.
104+
```
105+
106+
Each line identifies the exact JSON path where the violation occurred, making it straightforward to find and fix the issue.
107+
108+
---
109+
110+
## Versioning
111+
112+
Pin the action to a release tag or commit SHA to avoid unexpected breaking changes:
113+
114+
```yaml
115+
# Pin to a release tag (recommended)
116+
- uses: dbt-labs/terraform-dbtcloud-as-yaml/validate@v1
117+
118+
# Pin to a specific commit SHA for maximum reproducibility
119+
- uses: dbt-labs/terraform-dbtcloud-as-yaml/validate@2bb4e9e
120+
```
121+
122+
Avoid `@main` in production workflows — it tracks the development branch and may change at any time.
123+
124+
---
125+
126+
## Next Steps
127+
128+
<div class="grid cards" markdown>
129+
130+
- :material-pipe:{ .lg .middle } **CI/CD Integration**
131+
132+
---
133+
134+
Full CI and CD pipeline examples for GitHub Actions, GitLab, and Azure DevOps
135+
136+
[:octicons-arrow-right-24: CI/CD Guide](cicd.md)
137+
138+
- :material-file-code:{ .lg .middle } **YAML Schema Reference**
139+
140+
---
141+
142+
Full field reference for `dbt-config.yml`
143+
144+
[:octicons-arrow-right-24: YAML Schema](../configuration/yaml-schema.md)
145+
146+
- :material-security:{ .lg .middle } **Best Practices**
147+
148+
---
149+
150+
Secure and reliable deployments
151+
152+
[:octicons-arrow-right-24: Best Practices](best-practices.md)
153+
154+
</div>

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ nav:
7676
- Project Repository: reference/module-project_repository.md
7777
- Guides:
7878
- CI/CD Integration: guides/cicd.md
79+
- YAML Validation Action: guides/yaml-validation.md
7980
- Best Practices: guides/best-practices.md
8081
- Troubleshooting: guides/troubleshooting.md
8182

‎validate/action.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: "Validate dbt Cloud YAML"
2+
description: >
3+
Validates a dbt Cloud YAML configuration file against the dbt Cloud
4+
Terraform YAML schema (v1) before running Terraform or requiring any
5+
dbt Cloud credentials.
6+
7+
branding:
8+
icon: check-circle
9+
color: green
10+
11+
inputs:
12+
file:
13+
description: "Path to the dbt Cloud YAML configuration file to validate."
14+
required: false
15+
default: "dbt-config.yml"
16+
17+
runs:
18+
using: composite
19+
steps:
20+
- name: Set up Python
21+
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
22+
with:
23+
python-version: "3.x"
24+
25+
- name: Install check-jsonschema
26+
shell: bash
27+
run: pip install --quiet "check-jsonschema==0.37.1"
28+
29+
- name: Validate YAML against schema
30+
shell: bash
31+
env:
32+
INPUT_FILE: ${{ inputs.file }}
33+
run: |
34+
SCHEMA="${{ github.action_path }}/../schemas/v1.json"
35+
if [ ! -f "${SCHEMA}" ]; then
36+
echo "Error: schema file not found at '${SCHEMA}'."
37+
exit 1
38+
fi
39+
echo "Validating '${INPUT_FILE}' against dbt Cloud YAML schema v1..."
40+
check-jsonschema --schemafile "${SCHEMA}" "${INPUT_FILE}"
41+
echo "Validation passed."

‎validate/tests/invalid.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# yaml-language-server: $schema=https://raw.githubusercontent.com/dbt-labs/terraform-dbtcloud-as-yaml/refs/heads/main/schemas/v1.json
2+
# Invalid dbt Cloud YAML: missing required 'version' field and account.host_url.
3+
# Used to test that the validate GitHub Action correctly rejects bad input.
4+
5+
account:
6+
name: Broken-Account

‎validate/tests/valid.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# yaml-language-server: $schema=https://raw.githubusercontent.com/dbt-labs/terraform-dbtcloud-as-yaml/refs/heads/main/schemas/v1.json
2+
# Minimal valid dbt Cloud YAML used to test the validate GitHub Action.
3+
4+
version: 1
5+
account:
6+
name: Test-Account
7+
host_url: https://cloud.getdbt.com
8+
9+
globals:
10+
connections:
11+
- key: test_wh
12+
name: Test-Warehouse
13+
type: snowflake
14+
protected: true
15+
details:
16+
account: xy12345
17+
database: ANALYTICS
18+
warehouse: COMPUTE_WH
19+
20+
projects:
21+
- name: MyTestProject
22+
key: my_project
23+
repository:
24+
remote_url: acme/analytics
25+
environments:
26+
- name: Production
27+
key: prod
28+
type: deployment
29+
protected: true
30+
connection: test_wh
31+
credential:
32+
token_name: SNOWFLAKE_PASSWORD
33+
schema: ANALYTICS
34+
jobs:
35+
- name: Daily-Run
36+
key: daily_run
37+
environment_key: prod
38+
execute_steps:
39+
- dbt build
40+
triggers:
41+
schedule: true
42+
github_webhook: false
43+
git_provider_webhook: false
44+
on_merge: false
45+
schedule_type: days_of_week
46+
schedule_days: [0, 1, 2, 3, 4]
47+
schedule_hours: [6]

0 commit comments

Comments
 (0)