Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
d6f8543
update ignore duckdb files and add note about private key format
Jul 2, 2025
a309754
add genai prompt and update dbt_api script to pull file
Jul 4, 2025
c5b1efb
add info on using dbt fusion
Jul 4, 2025
86ca91a
cleanup sample script
Jul 9, 2025
a0874cc
cleanup dbt-api sample script
Jul 11, 2025
278a185
add GenAI to readme.
Jul 18, 2025
7b2329e
Merge branch 'dbt_fusion_testing' into feature/improve_gh_actions
Jul 18, 2025
50a728e
add script to generate airflow dag
Jul 19, 2025
2de7caa
update pre-commit hooks and dbt version
Jul 21, 2025
9d54e44
add 3rd step for ci/cd and branch name validations
Aug 18, 2025
cee0ffb
Merge branch 'main' into feature/improve_gh_actions
Aug 18, 2025
fc38493
update branch validation script
Aug 19, 2025
46f6bd5
fix env vars
Aug 19, 2025
8585f2c
update branch validation
Aug 19, 2025
3baad67
update validation messages
Aug 19, 2025
c1a2336
fix branch name
Aug 19, 2025
329da59
update messages
Aug 19, 2025
5cf6663
update comment
Aug 19, 2025
ec1432c
update workflow names
Aug 19, 2025
8e0abed
update workflow name
Aug 19, 2025
e46094f
Merge branch 'release/20250819' into feature/improve_gh_actions
noel Aug 19, 2025
7120cee
force a build
Aug 19, 2025
3d5a967
Merge branch 'feature/improve_gh_actions' of github.com:datacoves/bal…
Aug 19, 2025
ca3a9db
force build
Aug 19, 2025
502ad5c
force a build
Aug 19, 2025
fbe4702
change trigger
Aug 19, 2025
b36c346
Remove retry_dbt_failures_cosmos.py and transform/docs/* from git tra…
Aug 19, 2025
bc2ee28
fix workflow triggers
Aug 19, 2025
28c35a1
update trigger rules
Aug 19, 2025
f21ad76
Merge branch 'release/20250819' into feature/improve_gh_actions
noel Aug 19, 2025
24aff4e
force updates to transform and orchestrate
Aug 19, 2025
0a7b624
Merge branch 'feature/improve_gh_actions' of github.com:datacoves/bal…
Aug 19, 2025
77d1075
force a workflow run
Aug 19, 2025
4048770
fix trigger events
Aug 19, 2025
b749832
change workflow name
Aug 19, 2025
1fa9f4c
update workflow dependency
Aug 19, 2025
dc3b85b
fix workflow name
Aug 19, 2025
2297039
remove special chars from workflow name
Aug 19, 2025
dbd4739
merge branch validation into checks
Aug 19, 2025
3f189fc
cleanup matching rules
Aug 19, 2025
4689e29
remove if condition
Aug 20, 2025
ff020ab
add a comment to the PR with status
Aug 20, 2025
e64b9c8
fix yml typo
Aug 20, 2025
e826edc
fix comment
Aug 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 0 additions & 24 deletions .github/workflows/05_branch_validation.yml

This file was deleted.

37 changes: 35 additions & 2 deletions .github/workflows/10_feature_airflow_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ name: "10: 🎯 Airflow Validations"
on: # yamllint disable-line rule:truthy
pull_request:
branches:
- 'feature/**'
- 'release/**/*'
paths:
- orchestrate/*
- orchestrate/**/*

# Allows you to run this workflow manually from the Actions tab
Expand All @@ -17,6 +16,23 @@ concurrency:
cancel-in-progress: true

jobs:
validate-branch:
runs-on: ubuntu-latest
name: Validate Branch Names and Merge Rules and assure branch has changes from main

env:
SOURCE_BRANCH: ${{ github.head_ref }}
TARGET_BRANCH: ${{ github.base_ref }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Validate current branch
run: automate/git/branch_validator.py

airflow:
name: Pull Request Airflow Tests
runs-on: ubuntu-latest
Expand Down Expand Up @@ -64,3 +80,20 @@ jobs:
filePath: ${{ env.OUTPUT_FILE }}
comment_tag: Custom Tests
GITHUB_TOKEN: ${{ github.token }}

airflow-job-status:
runs-on: ubuntu-latest
needs: [airflow]
if: always()
steps:
- name: Comment PR with Airflow status
uses: thollander/actions-comment-pull-request@v2
with:
message: |
## 🎯 Airflow Workflow Status
- **Job**: Pull Request Airflow Tests → **${{ needs.airflow.result }}**

${{ needs.airflow.result == 'skipped' && '⏭️ _Skipped because no orchestrate/ files were changed_' || '' }}
${{ needs.airflow.result == 'success' && '✅ _All Airflow validations passed_' || '' }}
${{ needs.airflow.result == 'failure' && '❌ _Airflow validations failed_' || '' }}
comment_tag: airflow-workflow-status
37 changes: 35 additions & 2 deletions .github/workflows/10_feature_dbt_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ name: "10: 🧪 dbt Validations & Governance Checks"
on: # yamllint disable-line rule:truthy
pull_request:
branches:
- 'feature/**'
- 'release/**/*'
paths:
- transform/*
- transform/**/*

# Allows you to run this workflow manually from the Actions tab
Expand All @@ -17,6 +16,23 @@ concurrency:
cancel-in-progress: true

jobs:
validate-branch:
runs-on: ubuntu-latest
name: Validate Branch Names and Merge Rules and assure branch has changes from main

env:
SOURCE_BRANCH: ${{ github.head_ref }}
TARGET_BRANCH: ${{ github.base_ref }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Validate current branch
run: automate/git/branch_validator.py

dbt:
name: Pull Request dbt Tests
runs-on: ubuntu-latest
Expand Down Expand Up @@ -123,3 +139,20 @@ jobs:
- name: Drop PR database on Failure to grant security access
if: always() && (env.DATACOVES__DROP_DB_ON_FAIL == 'true') && (steps.grant-access-to-database.outcome == 'failure')
run: "dbt --no-write-json run-operation drop_recreate_db --args '{db_name: ${{env.DATACOVES__MAIN__DATABASE}}, recreate: False}'" # yamllint disable-line rule:line-length

dbt-job-status:
runs-on: ubuntu-latest
needs: [dbt]
if: always()
steps:
- name: Comment PR with dbt status
uses: thollander/actions-comment-pull-request@v2
with:
message: |
## 🧪 dbt Workflow Status
- **Job**: Pull Request dbt Tests → **${{ needs.dbt.result }}**

${{ needs.dbt.result == 'skipped' && '⏭️ _Skipped because no transform/ files were changed_' || '' }}
${{ needs.dbt.result == 'success' && '✅ _All dbt validations passed_' || '' }}
${{ needs.dbt.result == 'failure' && '❌ _dbt validations failed_' || '' }}
comment_tag: dbt-workflow-status
65 changes: 0 additions & 65 deletions .github/workflows/10_integrate_airflow_changes.yml

This file was deleted.

123 changes: 0 additions & 123 deletions .github/workflows/10_integrate_dbt_changes.yml

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
name: "20: 🗑️ Drop test databases"
name: "15: 🗑️ Drop test databases"

on: # yamllint disable-line rule:truthy
pull_request:
branches:
- 'feature/**'
- 'release/**/*'
types:
- closed

Expand Down
60 changes: 0 additions & 60 deletions .github/workflows/20_drop_integration_db.yml

This file was deleted.

Loading
Loading