Skip to content

Commit 4db4c5e

Browse files
authored
17 build starter set of dbt cloud topologies (#28)
* add basic topology * bump for vulnerabilities * bump tf lock * bump module lock files * update docs
1 parent e2eefdf commit 4db4c5e

89 files changed

Lines changed: 2322 additions & 1023 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,19 @@ on:
66
- "**.tf"
77
- "tests/**"
88
- "schemas/**"
9+
- "docs/reference/**"
10+
- ".terraform-docs.yml"
911
- ".github/workflows/ci.yml"
10-
- "!examples/**"
12+
- "!topologies/**"
1113
push:
1214
branches: [main]
1315
paths:
1416
- "**.tf"
1517
- "tests/**"
1618
- "schemas/**"
17-
- "!examples/**"
19+
- "docs/reference/**"
20+
- ".terraform-docs.yml"
21+
- "!topologies/**"
1822

1923
permissions:
2024
contents: read
@@ -56,3 +60,30 @@ jobs:
5660
5761
- name: Test (mock providers)
5862
run: terraform test
63+
64+
docs:
65+
name: Docs up to date
66+
runs-on: ubuntu-latest
67+
env:
68+
TERRAFORM_DOCS_VERSION: v0.20.0
69+
steps:
70+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
71+
72+
- name: Install terraform-docs
73+
run: |
74+
wget -q -O /tmp/terraform-docs.tar.gz \
75+
https://github.com/terraform-docs/terraform-docs/releases/download/${TERRAFORM_DOCS_VERSION}/terraform-docs-${TERRAFORM_DOCS_VERSION}-linux-amd64.tar.gz
76+
tar -xzf /tmp/terraform-docs.tar.gz -C /tmp
77+
sudo mv /tmp/terraform-docs /usr/local/bin/
78+
chmod +x /usr/local/bin/terraform-docs
79+
80+
- name: Regenerate docs
81+
run: make docs
82+
83+
- name: Check for drift
84+
run: |
85+
if ! git diff --exit-code docs/reference/; then
86+
echo ""
87+
echo "Docs are out of date. Run 'make docs' locally and commit the result."
88+
exit 1
89+
fi

.github/workflows/deploy_docs.yml

Lines changed: 3 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ on:
1313
- '*.tf'
1414
- 'modules/**'
1515
- 'schemas/**'
16-
- 'examples/**'
16+
- 'topologies/**'
1717
- '.github/workflows/deploy_docs.yml'
1818

1919
pull_request:
@@ -24,7 +24,7 @@ on:
2424
- '*.tf'
2525
- 'modules/**'
2626
- 'schemas/**'
27-
- 'examples/**'
27+
- 'topologies/**'
2828

2929
# Allows you to run this workflow manually from the Actions tab
3030
workflow_dispatch:
@@ -77,15 +77,7 @@ jobs:
7777
terraform-docs --version
7878
7979
- name: Generate Terraform documentation
80-
run: |
81-
# Generate main module docs
82-
terraform-docs markdown table . > docs/reference/terraform.md
83-
84-
# Generate submodule docs
85-
for dir in modules/*/; do
86-
module_name=$(basename $dir)
87-
terraform-docs markdown table $dir > docs/reference/module-${module_name}.md
88-
done
80+
run: make docs
8981

9082
- name: Setup Pages
9183
id: pages

.github/workflows/release.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# release.yml — triggered by a version tag (e.g. git tag v1.2.3 && git push --tags).
22
#
33
# Produces two release assets:
4-
# starter.tar.gz — the examples/basic/ directory, ready to extract
4+
# starter.tar.gz — the topologies/basic/ directory, ready to extract
55
# install.sh — bootstrap script that downloads starter.tar.gz
66

77
name: Release
@@ -24,7 +24,7 @@ jobs:
2424
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
2525

2626
- name: Package starter
27-
run: tar -czf starter.tar.gz -C examples/basic .
27+
run: tar -czf starter.tar.gz -C topologies/basic .
2828

2929
- name: Create release with assets
3030
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2

.github/workflows/test.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,15 @@ on:
1313
- "tests/**"
1414
- "schemas/**"
1515
- ".github/workflows/test.yml"
16-
- "!examples/**"
16+
- "!topologies/**"
1717
pull_request:
1818
paths:
1919
- "**.tf"
2020
- "modules/**"
2121
- "tests/**"
2222
- "schemas/**"
2323
- ".github/workflows/test.yml"
24-
- "!examples/**"
24+
- "!topologies/**"
2525

2626
permissions:
2727
contents: read

.pre-commit-config.yaml

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,23 @@
11
repos:
22
# Terraform formatting and documentation
3-
- repo: https://github.com/terraform-docs/terraform-docs
4-
rev: v0.16.0
3+
- repo: local
54
hooks:
65
- id: terraform-docs-go
76
name: terraform-docs
87
description: Generate terraform documentation
98
entry: terraform-docs -c .terraform-docs.yml
10-
language: golang
9+
language: system
1110
files: ^[^/]*\.tf$
1211
exclude: ^\.terraform/.*$
13-
args: [--sort-by-required]
1412

1513
# Terraform linting
16-
- repo: https://github.com/terraform-linters/tflint
17-
rev: v0.50.3
14+
- repo: local
1815
hooks:
1916
- id: tflint
2017
name: tflint
2118
description: Run tflint to check terraform code quality
2219
entry: tflint
23-
language: golang
20+
language: system
2421
files: \.tf$
2522
exclude: ^\.terraform/.*$
2623

.terraform-docs.yml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
formatter: "markdown table"
2+
3+
output:
4+
file: docs/reference/terraform.md
5+
mode: replace
6+
7+
sort:
8+
enabled: true
9+
by: required
10+
11+
settings:
12+
anchor: true
13+
color: true
14+
default: true
15+
description: true
16+
escape: true
17+
indent: 2
18+
required: true
19+
sensitive: true
20+
type: true

.terraform.lock.hcl

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Makefile

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,15 @@ test: ## Run terraform test with mock providers (no credentials needed)
2121
test-integration: ## Run Go integration tests against a real dbt Cloud account (requires DBT_CLOUD_ACCOUNT_ID and DBT_CLOUD_TOKEN)
2222
cd test && RUN_INTEGRATION_TESTS=1 go test -v -timeout 30m -run Integration ./...
2323

24-
docs: ## Regenerate terraform-docs for all modules
25-
pre-commit run terraform-docs-go --all-files
24+
docs: ## Regenerate terraform-docs for root module and all submodules
25+
terraform-docs -c .terraform-docs.yml .
26+
@for dir in modules/*/; do \
27+
module=$$(basename $$dir); \
28+
terraform-docs markdown table \
29+
--output-file "../../docs/reference/module-$${module}.md" \
30+
--output-mode replace \
31+
"$$dir"; \
32+
done
2633

2734
pre-commit: ## Run all pre-commit hooks on staged files
2835
pre-commit run

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# terraform-dbtcloud-as-yaml
22

33
[![Terraform Version](https://img.shields.io/badge/terraform-%3E%3D%201.0-blue?logo=terraform)](https://www.terraform.io)
4-
[![dbt Cloud Provider](https://img.shields.io/badge/dbt--cloud--provider-%3E%3D%201.8-blue)](https://registry.terraform.io/providers/dbt-labs/dbtcloud/latest)
4+
[![dbt Cloud Provider](https://img.shields.io/badge/dbt--cloud--provider-%3E%3D%201.9-blue)](https://registry.terraform.io/providers/dbt-labs/dbtcloud/latest)
55
[![License](https://img.shields.io/badge/license-Apache%202.0-green)](./LICENSE)
66

77
Managing dbt Cloud infrastructure shouldn't require deep Terraform expertise. Whether you've never written HCL or just don't want to untangle the provider's resource model yourself, this module gives you a single YAML file that maps to how dbt Platform actually works — projects, environments, jobs, and connections — with relationship wiring, credential handling, and production safeguards built in.
@@ -12,7 +12,7 @@ Managing dbt Cloud infrastructure shouldn't require deep Terraform expertise. Wh
1212
curl -fsSL https://github.com/dbt-labs/terraform-dbtcloud-as-yaml/releases/latest/download/install.sh | bash
1313
```
1414

15-
This downloads the [examples/basic/](examples/basic/) starter into `./my-dbt-cloud`. No npm, no git magic — just curl and tar. To use a different directory name: `curl -fsSL ... | bash -s -- my-project`.
15+
This downloads the [topologies/basic/](topologies/basic/) starter into `./my-dbt-cloud`. No npm, no git magic — just curl and tar. To use a different directory name: `curl -fsSL ... | bash -s -- my-project`.
1616

1717
Then:
1818

@@ -23,7 +23,7 @@ cp .env.example .env # fill in your dbt Cloud credentials
2323
source .env && terraform init && terraform apply
2424
```
2525

26-
See [examples/basic/README.md](examples/basic/README.md) for a full walkthrough.
26+
See [topologies/basic/README.md](topologies/basic/README.md) for a full walkthrough.
2727

2828
---
2929

@@ -327,7 +327,7 @@ Terraform credentials.
327327
## Documentation
328328

329329
- **Hosted docs (GitHub Pages):** [dbt-labs.github.io/terraform-dbtcloud-as-yaml](https://dbt-labs.github.io/terraform-dbtcloud-as-yaml/)
330-
- [examples/basic/](examples/basic/) — clone-and-go starter with step-by-step README
330+
- [topologies/basic/](topologies/basic/) — clone-and-go starter with step-by-step README
331331
- [docs/configuration/yaml-schema.md](docs/configuration/yaml-schema.md) — full YAML field reference
332332
- [docs/guides/cicd.md](docs/guides/cicd.md) — CI/CD setup (GitHub Actions, etc.)
333333
- [docs/guides/best-practices.md](docs/guides/best-practices.md) — patterns and recommendations

docs/getting-started/examples.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The simplest possible setup to get started.
1717
### Directory Structure
1818

1919
```
20-
examples/basic/
20+
topologies/basic/
2121
├── main.tf # Terraform module call
2222
├── variables.tf # Input variables
2323
├── dbt-config.yml # dbt Cloud configuration
@@ -31,7 +31,7 @@ examples/basic/
3131
### Try It Out
3232

3333
```bash
34-
cd examples/basic
34+
cd topologies/basic
3535
cp .env.example .env
3636
# Edit .env with your credentials
3737
source .env
@@ -40,7 +40,7 @@ terraform plan
4040
terraform apply
4141
```
4242

43-
[:material-github: View Source](https://github.com/dbt-labs/terraform-dbtcloud-as-yaml/tree/main/examples/basic){ .md-button }
43+
[:material-github: View Source](https://github.com/dbt-labs/terraform-dbtcloud-as-yaml/tree/main/topologies/basic){ .md-button }
4444

4545
---
4646

@@ -264,7 +264,7 @@ projects:
264264
265265
## CI/CD with GitHub Actions
266266
267-
Use the two-workflow pattern from `examples/basic/.github/workflows/`:
267+
Use the two-workflow pattern from `topologies/basic/.github/workflows/`:
268268

269269
- `ci.yml` — validates and plans on every PR, posts plan as a comment (updates existing comment on re-push)
270270
- `cd.yml` — applies on merge to main, with optional approval gate via GitHub Environments

0 commit comments

Comments
 (0)