Skip to content

Commit 9494dfd

Browse files
xophamxopham
authored
ci: segment version-update PR creation (#19605)
## Description Segment the version-generation workflows so dependency installation and generated-file production run separately from pull request publication. The generation jobs now pass a short-lived patch artifact to a minimal publication job. Before creating a pull request, that job verifies the patch only touches the expected generated files and preserves regular file modes. Each workflow also has a dedicated, tightly matched STS policy. This improves the credential boundary while preserving the existing generated pull request behavior. Tracks [APMSP-3144](https://datadoghq.atlassian.net/browse/APMSP-3144). ## Testing - `scripts/lint checks` - Parsed all changed workflow, composite-action, and STS policy files as YAML - Verified the embedded Bash syntax - Exercised the patch gate in an isolated Git repository: - accepted and round-tripped an allowed `supported_versions.json` change - rejected an unexpected `.gitlab-ci.yml` change - `scripts/run-tests --list ...` found no applicable product test suites for these workflow-only changes ## Risks The artifact handoff and cross-job output propagation could affect the generated PR workflows. The patch gate checks applicability, changed paths, file modes, non-empty content, and whitespace before the publication credential is requested. ## Additional Notes No customer-facing behavior changes; no release note is needed. [APMSP-3144]: https://datadoghq.atlassian.net/browse/APMSP-3144?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: christoph.hamsen <christoph.hamsen@datadoghq.com>
1 parent 2d5e7cb commit 9494dfd

7 files changed

Lines changed: 291 additions & 31 deletions

‎.github/actions/generated-change-patch/action.yml‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
name: Generated change patch
2+
description: Create or safely apply a patch containing generated version updates
3+
4+
inputs:
5+
mode:
6+
description: Whether to create or apply the patch
7+
required: true
8+
patch-file:
9+
description: Path to the patch artifact
10+
required: true
11+
profile:
12+
description: Allowlist of generated paths to enforce
13+
required: true
14+
15+
outputs:
16+
changed:
17+
description: Whether the generated patch contains changes
18+
value: ${{ steps.patch.outputs.changed }}
19+
20+
runs:
21+
using: composite
22+
steps:
23+
# AIDEV-NOTE: Keep patch application and validation in this single shell step,
24+
# and do not add repository-mutating steps between this action and PR creation.
25+
# The patch is untrusted data and must not replace code that is executed later.
26+
- id: patch
27+
shell: bash
28+
env:
29+
MODE: ${{ inputs.mode }}
30+
PATCH_FILE: ${{ inputs.patch-file }}
31+
PROFILE: ${{ inputs.profile }}
32+
run: |
33+
set -euo pipefail
34+
35+
validate_path() {
36+
local path=$1
37+
38+
case "$PROFILE" in
39+
package-versions)
40+
if [[ "$path" =~ ^\.riot/requirements/[^/]+\.txt$ ]] ||
41+
[[ "$path" == "supported_versions.json" ]] ||
42+
[[ "$path" == "scripts/integration_registry/registry.yaml" ]]; then
43+
return
44+
fi
45+
;;
46+
supported-versions)
47+
if [[ "$path" == "supported_versions.json" ]]; then
48+
return
49+
fi
50+
;;
51+
*)
52+
printf '::error::Unknown generated-change profile: %q\n' "$PROFILE"
53+
exit 1
54+
;;
55+
esac
56+
57+
printf '::error::Unexpected generated change: %q\n' "$path"
58+
exit 1
59+
}
60+
61+
validate_mode() {
62+
local path=$1
63+
local mode
64+
mode=$(git ls-files -s -- "$path" | awk '{print $1}')
65+
if [[ -n "$mode" && "$mode" != "100644" ]]; then
66+
printf '::error::Unexpected file mode %q for %q\n' "$mode" "$path"
67+
exit 1
68+
fi
69+
}
70+
71+
if [[ "$MODE" == "create" ]]; then
72+
while IFS= read -r -d '' path; do
73+
validate_path "$path"
74+
git add --intent-to-add -- "$path"
75+
done < <(git ls-files --others --exclude-standard -z)
76+
77+
if git diff --quiet HEAD --; then
78+
echo "changed=false" >> "$GITHUB_OUTPUT"
79+
exit 0
80+
fi
81+
82+
while IFS= read -r -d '' path; do
83+
validate_path "$path"
84+
validate_mode "$path"
85+
done < <(git diff --name-only -z HEAD --)
86+
87+
# Keep generated artifacts clean before they cross into the PR-creation job.
88+
git diff --check HEAD --
89+
git diff --binary --full-index --no-ext-diff HEAD -- > "$PATCH_FILE"
90+
echo "changed=true" >> "$GITHUB_OUTPUT"
91+
exit 0
92+
fi
93+
94+
if [[ "$MODE" == "apply" ]]; then
95+
git apply --check --index "$PATCH_FILE"
96+
git apply --index "$PATCH_FILE"
97+
98+
changed=false
99+
while IFS= read -r -d '' path; do
100+
changed=true
101+
validate_path "$path"
102+
validate_mode "$path"
103+
done < <(git diff --cached --name-only -z)
104+
105+
if [[ "$changed" != "true" ]]; then
106+
echo "::error::Generated patch contains no changes"
107+
exit 1
108+
fi
109+
110+
git diff --cached --check
111+
echo "changed=true" >> "$GITHUB_OUTPUT"
112+
exit 0
113+
fi
114+
115+
printf '::error::Unknown generated-change mode: %q\n' "$MODE"
116+
exit 1

‎.github/chainguard/self.generate-package-versions.create-pr.sts.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
1+
# Policy for: create-pull-request in .github/workflows/generate-package-versions.yml
12
issuer: https://token.actions.githubusercontent.com
23

34
subject: repo:DataDog/dd-trace-py:ref:refs/heads/main
45

56
claim_pattern:
67
event_name: (workflow_dispatch|schedule)
8+
job_workflow_ref: DataDog/dd-trace-py/\.github/workflows/generate-package-versions\.yml@refs/heads/main
79
ref: refs/heads/main
8-
ref_protected: "true"
9-
job_workflow_ref: DataDog/dd-trace-py/\.github/workflows/(generate-package-versions|update-package-version)\.yml@refs/heads/main
10+
repository: DataDog/dd-trace-py
1011

1112
permissions:
1213
contents: write
1314
pull_requests: write
14-

‎.github/chainguard/self.generate-supported-versions.create-pr.sts.yaml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
1+
# Policy for: create-pull-request in .github/workflows/generate-supported-versions.yml
12
issuer: https://token.actions.githubusercontent.com
23

34
subject: repo:DataDog/dd-trace-py:ref:refs/heads/main
45

56
claim_pattern:
67
event_name: workflow_dispatch
7-
ref: refs/heads/main
8-
ref_protected: "true"
98
job_workflow_ref: DataDog/dd-trace-py/\.github/workflows/generate-supported-versions\.yml@refs/heads/main
9+
ref: refs/heads/main
10+
repository: DataDog/dd-trace-py
1011

1112
permissions:
1213
contents: write
13-
pull_requests: write
14+
pull_requests: write

‎.github/chainguard/self.update-package-version.create-pr.sts.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Policy for: create-pull-request in .github/workflows/update-package-version.yml
2+
issuer: https://token.actions.githubusercontent.com
3+
4+
subject: repo:DataDog/dd-trace-py:ref:refs/heads/main
5+
6+
claim_pattern:
7+
event_name: workflow_dispatch
8+
job_workflow_ref: DataDog/dd-trace-py/\.github/workflows/update-package-version\.yml@refs/heads/main
9+
ref: refs/heads/main
10+
repository: DataDog/dd-trace-py
11+
12+
permissions:
13+
contents: write
14+
pull_requests: write

‎.github/workflows/generate-package-versions.yml‎

Lines changed: 56 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,13 @@ jobs:
1010
name: Update riot lockfiles
1111
if: github.event_name != 'schedule' || github.event.repository.fork == false
1212
runs-on: ubuntu-22.04
13+
outputs:
14+
changed: ${{ steps.patch.outputs.changed }}
15+
new_latest: ${{ steps.new-latest.outputs.new_latest }}
16+
venv_name: ${{ steps.new-latest.outputs.venv_name }}
1317
permissions:
1418
actions: read
15-
contents: write
16-
id-token: write
19+
contents: read
1720

1821
steps:
1922
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -104,7 +107,52 @@ jobs:
104107
id: new-latest
105108
run: |
106109
NEW_LATEST=$(python scripts/get_latest_version.py ${{ env.VENV_NAME }})
107-
echo "NEW_LATEST=$NEW_LATEST" >> $GITHUB_ENV
110+
echo "new_latest=$NEW_LATEST" >> "$GITHUB_OUTPUT"
111+
echo "venv_name=$VENV_NAME" >> "$GITHUB_OUTPUT"
112+
113+
- name: Prepare generated changes
114+
id: patch
115+
uses: ./.github/actions/generated-change-patch
116+
with:
117+
mode: create
118+
patch-file: generated.patch
119+
profile: package-versions
120+
121+
- name: Upload generated changes
122+
if: steps.patch.outputs.changed == 'true'
123+
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
124+
with:
125+
name: generated-package-versions
126+
path: generated.patch
127+
retention-days: 1
128+
129+
create-pull-request:
130+
name: Create pull request
131+
needs: update-riot-lockfiles
132+
if: needs.update-riot-lockfiles.outputs.changed == 'true'
133+
runs-on: ubuntu-22.04
134+
permissions:
135+
actions: read
136+
contents: read
137+
id-token: write
138+
139+
steps:
140+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
141+
with:
142+
persist-credentials: false
143+
144+
- name: Download generated changes
145+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
146+
with:
147+
name: generated-package-versions
148+
path: ${{ runner.temp }}/generated-package-versions
149+
150+
- name: Apply and validate generated changes
151+
uses: ./.github/actions/generated-change-patch
152+
with:
153+
mode: apply
154+
patch-file: ${{ runner.temp }}/generated-package-versions/generated.patch
155+
profile: package-versions
108156

109157
- uses: DataDog/dd-octo-sts-action@96a25462dbcb10ebf0bfd6e2ccc917d2ab235b9a # v1.0.4
110158
id: octo-sts
@@ -118,14 +166,14 @@ jobs:
118166
with:
119167
token: ${{ steps.octo-sts.outputs.token }}
120168
sign-commits: true
121-
branch: "upgrade-latest-${{ env.VENV_NAME }}-version"
169+
branch: "upgrade-latest-${{ needs.update-riot-lockfiles.outputs.venv_name }}-version"
122170
commit-message: "Update package version"
123171
delete-branch: true
124172
base: main
125-
title: "chore: update ${{ env.VENV_NAME }} latest version to ${{ env.NEW_LATEST }}"
173+
title: "chore: update ${{ needs.update-riot-lockfiles.outputs.venv_name }} latest version to ${{ needs.update-riot-lockfiles.outputs.new_latest }}"
126174
labels: changelog/no-changelog
127175
body: |
128-
Update ${{ env.VENV_NAME }} lockfiles and dependency package lockfiles.
176+
Update ${{ needs.update-riot-lockfiles.outputs.venv_name }} lockfiles and dependency package lockfiles.
129177
This performs the following updates:
130-
1) Some ${{ env.VENV_NAME }} lockfiles use ${{ env.VENV_NAME }} `latest`. This will update ${{ env.VENV_NAME }} and dependencies.
131-
2) Some ${{ env.VENV_NAME }} lockfiles use a pinned (non-latest) version of ${{ env.VENV_NAME }}, but require the `latest` version of another package. This will update all such packages.
178+
1) Some ${{ needs.update-riot-lockfiles.outputs.venv_name }} lockfiles use ${{ needs.update-riot-lockfiles.outputs.venv_name }} `latest`. This will update ${{ needs.update-riot-lockfiles.outputs.venv_name }} and dependencies.
179+
2) Some ${{ needs.update-riot-lockfiles.outputs.venv_name }} lockfiles use a pinned (non-latest) version of ${{ needs.update-riot-lockfiles.outputs.venv_name }}, but require the `latest` version of another package. This will update all such packages.

‎.github/workflows/generate-supported-versions.yml‎

Lines changed: 49 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@ jobs:
77
generate-supported-versions:
88
name: Generate supported integration versions
99
runs-on: ubuntu-22.04
10+
outputs:
11+
changed: ${{ steps.patch.outputs.changed }}
1012
permissions:
1113
actions: read
12-
contents: write
13-
id-token: write
14+
contents: read
1415

1516
steps:
1617
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -78,7 +79,52 @@ jobs:
7879
print(json.dumps(supported_versions[:3], indent=2))
7980
PY
8081
81-
- run: git diff
82+
- name: Preview changes
83+
run: git diff
84+
85+
- name: Prepare generated changes
86+
id: patch
87+
uses: ./.github/actions/generated-change-patch
88+
with:
89+
mode: create
90+
patch-file: generated.patch
91+
profile: supported-versions
92+
93+
- name: Upload generated changes
94+
if: steps.patch.outputs.changed == 'true'
95+
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
96+
with:
97+
name: generated-supported-versions
98+
path: generated.patch
99+
retention-days: 1
100+
101+
create-pull-request:
102+
name: Create pull request
103+
needs: generate-supported-versions
104+
if: needs.generate-supported-versions.outputs.changed == 'true'
105+
runs-on: ubuntu-22.04
106+
permissions:
107+
actions: read
108+
contents: read
109+
id-token: write
110+
111+
steps:
112+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
113+
with:
114+
persist-credentials: false
115+
116+
- name: Download generated changes
117+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
118+
with:
119+
name: generated-supported-versions
120+
path: ${{ runner.temp }}/generated-supported-versions
121+
122+
- name: Apply and validate generated changes
123+
uses: ./.github/actions/generated-change-patch
124+
with:
125+
mode: apply
126+
patch-file: ${{ runner.temp }}/generated-supported-versions/generated.patch
127+
profile: supported-versions
82128

83129
- uses: DataDog/dd-octo-sts-action@96a25462dbcb10ebf0bfd6e2ccc917d2ab235b9a # v1.0.4
84130
id: octo-sts

0 commit comments

Comments
 (0)