Skip to content

Commit a8e74a9

Browse files
danielfromearthdanielfromearth
committed
feat: Complete CI/CD workflow restructure and GitHub Actions migration
- Restructure GitHub Actions workflows for better organization and efficiency - Add dedicated PR validation workflow (pr-checks.yml) with security scanning, linting, type checks, and CHANGELOG validation - Create reusable workflows for unit tests, integration tests, and Docker builds - Add publish workflow for release automation to PyPI - Implement auto-bump workflow for version management after release branch creation - Modify pull request workflows for both internal and external contributors - Enhance version bumping strategy - Only bump versions on PR merges, not direct pushes - Add scripts for automated version management (version_bump.sh, verify_tag.sh) - Separate version bumping from build/publish steps - Improve testing workflows - Rename and consolidate test workflows (unit-tests.yml, integration-tests.yml) - Add timeout configurations and dependency caching - Use frozen dependencies for reproducible builds - Add comprehensive documentation - Create CI-CD-README.md with detailed workflow explanations - Add CI_CD_ARCHITECTURE_DIAGRAM.md with Mermaid diagrams - Update HOW_TO_RELEASE.md with additional info about release process - Publishing improvements - Separate publishing to Test PyPI (release branches) and production PyPI (releases) - Add Docker image building for different environments (sit, uat, ops) - Implement venue-based deployments - Configuration updates - Update dependabot configuration with better grouping and labeling - Enhance PR template with additional checklist items - Remove poetry.lock (migrated to uv)
1 parent 354db4e commit a8e74a9

19 files changed

Lines changed: 1950 additions & 3470 deletions

‎.github/dependabot.yml‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,18 @@ updates:
77
groups:
88
pip-dependencies:
99
patterns:
10-
- "*"
11-
# Raise pull requests for version updates
12-
# to pip against the `develop` branch
10+
- "*" # Groups all dependencies
1311
target-branch: "develop"
12+
open-pull-requests-limit: 10
13+
labels:
14+
- "dependencies"
15+
1416
- package-ecosystem: "github-actions"
1517
directory: "/"
1618
schedule:
1719
interval: "monthly"
1820
groups:
1921
gha-dependencies:
2022
patterns:
21-
- "*"
23+
- "*" # Groups all GitHub Actions
2224
target-branch: "develop"

‎.github/pull_request_template.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,8 @@ _Summarize testing and verification you've done. This includes unit tests or tes
1313
_Explain how this change was integration tested. Provide screenshots or logs if appropriate. An example of this would be a local Harmony deployment._
1414

1515
## PR Acceptance Checklist
16-
* [ ] Unit tests added/updated and passing.
17-
* [ ] Integration testing
16+
* [ ] Unit tests added/updated and passing
17+
* [ ] Integration testing completed
1818
* [ ] `CHANGELOG.md` updated
19-
* [ ] Documentation updated (if needed).
19+
* [ ] Documentation updated (if needed)
20+
* [ ] Breaking changes documented (if any)

‎.github/workflows/auto-bump-develop.yml‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
name: Auto-bump develop after release branch created
2+
3+
on:
4+
create:
5+
workflow_dispatch:
6+
inputs:
7+
release_version:
8+
description: 'Release version (e.g., 1.11.0)'
9+
required: true
10+
type: string
11+
12+
env:
13+
PYTHON_VERSION: "3.12"
14+
15+
jobs:
16+
bump-develop:
17+
name: Bump develop to next minor version
18+
runs-on: ubuntu-latest
19+
# Only run for release branch creation or manual dispatch
20+
if: github.event_name == 'workflow_dispatch' || (github.event_name == 'create' && github.ref_type == 'branch' && startsWith(github.ref_name, 'release/'))
21+
timeout-minutes: 10
22+
permissions:
23+
contents: write
24+
25+
steps:
26+
- name: Checkout repository
27+
uses: actions/checkout@v6
28+
with:
29+
fetch-depth: 0
30+
31+
- name: Set up Python
32+
uses: actions/setup-python@v6
33+
with:
34+
python-version: ${{ env.PYTHON_VERSION }}
35+
36+
- name: Install bump-my-version
37+
run: pip install bump-my-version
38+
39+
- name: Configure git
40+
run: |
41+
git config user.name "stitchee-bot"
42+
git config user.email "stitchee@noreply.github.com"
43+
44+
- name: Extract release version
45+
id: release
46+
run: |
47+
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
48+
RELEASE_VERSION="${{ inputs.release_version }}"
49+
else
50+
# For create event, github.ref_name contains the branch name
51+
BRANCH="${{ github.ref_name }}"
52+
RELEASE_VERSION="${BRANCH#release/}"
53+
fi
54+
echo "version=$RELEASE_VERSION" >> $GITHUB_OUTPUT
55+
echo "📦 Release branch created for version: $RELEASE_VERSION"
56+
57+
- name: Bump develop to next minor
58+
run: |
59+
git fetch origin develop
60+
git checkout develop
61+
62+
CURRENT=$(bump-my-version show current_version)
63+
RELEASE_VERSION="${{ steps.release.outputs.version }}"
64+
65+
echo "Develop is at: $CURRENT"
66+
echo "Release version: $RELEASE_VERSION"
67+
68+
# Only bump if develop is still on the version being released
69+
if [[ "$CURRENT" == "${RELEASE_VERSION}"* ]]; then
70+
echo "✅ Develop needs to be bumped to next minor"
71+
72+
# Calculate next minor version
73+
MAJOR=$(echo "$RELEASE_VERSION" | cut -d. -f1)
74+
MINOR=$(echo "$RELEASE_VERSION" | cut -d. -f2)
75+
NEXT_MINOR=$((MINOR + 1))
76+
NEXT_VERSION="${MAJOR}.${NEXT_MINOR}.0a1"
77+
78+
echo "Bumping: $CURRENT → $NEXT_VERSION"
79+
80+
bump-my-version bump --new-version "$NEXT_VERSION" minor
81+
git commit -am "Start $NEXT_VERSION development [skip ci]"
82+
git push origin develop
83+
84+
echo "✅ Develop bumped to $NEXT_VERSION"
85+
else
86+
echo "ℹ️ Develop already at $CURRENT (not on $RELEASE_VERSION), skipping bump"
87+
fi

‎.github/workflows/docker-build.yml‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: Docker Build and Push
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
version:
7+
description: 'Version to build (e.g., 1.11.0, 1.11.0rc1, 1.11.0a5)'
8+
required: true
9+
type: string
10+
venue:
11+
description: 'Deployment venue (dev, sit, uat, ops)'
12+
required: true
13+
type: string
14+
push:
15+
description: 'Whether to push the image to registry'
16+
required: false
17+
type: boolean
18+
default: true
19+
workflow_dispatch:
20+
inputs:
21+
version:
22+
description: 'Version to build (e.g., 1.11.0, 1.11.0rc1, 1.11.0a5)'
23+
required: true
24+
type: string
25+
venue:
26+
description: 'Deployment venue (dev, sit, uat, ops)'
27+
required: true
28+
type: choice
29+
options:
30+
- dev
31+
- sit
32+
- uat
33+
- ops
34+
push:
35+
description: 'Whether to push the image to registry'
36+
required: false
37+
type: boolean
38+
default: true
39+
40+
env:
41+
REGISTRY: ghcr.io
42+
IMAGE_NAME: ${{ github.repository }}
43+
44+
jobs:
45+
docker:
46+
runs-on: ubuntu-latest
47+
permissions:
48+
contents: read
49+
packages: write
50+
51+
steps:
52+
- name: Checkout code
53+
uses: actions/checkout@v6
54+
55+
- name: Log in to Container Registry
56+
if: ${{ inputs.push }}
57+
uses: docker/login-action@v3
58+
with:
59+
registry: ${{ env.REGISTRY }}
60+
username: ${{ github.actor }}
61+
password: ${{ secrets.GITHUB_TOKEN }}
62+
63+
- name: Extract Docker metadata
64+
id: meta
65+
uses: docker/metadata-action@v5
66+
with:
67+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
68+
tags: |
69+
type=raw,value=${{ inputs.version }}
70+
type=raw,value=${{ inputs.venue }}
71+
type=raw,value=latest,enable=${{ inputs.venue == 'ops' }}
72+
73+
- name: Set up Docker Buildx
74+
uses: docker/setup-buildx-action@v3
75+
76+
- name: Build and push Docker image
77+
uses: docker/build-push-action@v6
78+
with:
79+
context: .
80+
file: Dockerfile
81+
push: ${{ inputs.push }}
82+
pull: true
83+
tags: ${{ steps.meta.outputs.tags }}
84+
labels: ${{ steps.meta.outputs.labels }}
85+
# Pull from multiple venue caches for better cache hit rates
86+
cache-from: |
87+
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache-${{ inputs.venue }}
88+
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache-sit
89+
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache-uat
90+
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache-${{ inputs.venue }},mode=max
91+
92+
- name: Output image tags
93+
run: |
94+
echo "### 🐳 Docker Image Published" >> $GITHUB_STEP_SUMMARY
95+
echo "" >> $GITHUB_STEP_SUMMARY
96+
echo "**Version:** \`${{ inputs.version }}\`" >> $GITHUB_STEP_SUMMARY
97+
echo "**Venue:** \`${{ inputs.venue }}\`" >> $GITHUB_STEP_SUMMARY
98+
echo "" >> $GITHUB_STEP_SUMMARY
99+
echo "**Tags:**" >> $GITHUB_STEP_SUMMARY
100+
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
101+
echo "${{ steps.meta.outputs.tags }}" >> $GITHUB_STEP_SUMMARY
102+
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY

.github/workflows/integration-test.yml renamed to .github/workflows/integration-tests.yml

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,15 @@
11
name: Integration Tests
22

33
on:
4-
# Allows you to run this workflow manually from the Actions tab
5-
workflow_dispatch:
4+
workflow_call:
65
secrets:
6+
codecov_token:
7+
required: true
78
DEK_EDL_USER:
89
required: true
910
DEK_EDL_PASSWORD:
1011
required: true
11-
codecov_token:
12-
required: true
13-
push:
14-
branches:
15-
- main
16-
- develop
17-
- release/**
18-
- feature/**
12+
workflow_dispatch:
1913

2014
# When this workflow is queued, automatically cancel any previous running
2115
# or pending jobs from the same branch
@@ -29,23 +23,25 @@ env:
2923
jobs:
3024
integration-tests:
3125
runs-on: ubuntu-latest
26+
timeout-minutes: 15
3227

3328
steps:
3429
- name: Retrieve repository
35-
uses: actions/checkout@v5
30+
uses: actions/checkout@v6
3631

3732
- name: Set up Python
3833
uses: actions/setup-python@v6
3934
with:
4035
python-version: ${{ env.PYTHON_VERSION }}
4136

4237
- name: Install uv
43-
uses: astral-sh/setup-uv@v5
38+
uses: astral-sh/setup-uv@v7
4439
with:
4540
enable-cache: true
41+
cache-dependency-path: "pyproject.toml"
4642

47-
- name: Install package
48-
run: uv sync --extra dev --extra integration
43+
- name: Install dependencies
44+
run: uv sync --extra dev --extra integration --frozen
4945

5046
- name: Test
5147
env:

0 commit comments

Comments
 (0)