Add self hosted runner for migration test GitHub Action workflow#14175
Add self hosted runner for migration test GitHub Action workflow#14175VenukshiMendis wants to merge 1 commit intowso2:masterfrom
Conversation
VenukshiMendis
requested review from
AnuGayan,
Arshardh,
chamilaadhi and
tharindu1st
as code owners
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughSummaryThis pull request updates the migration test GitHub Actions workflow to run key jobs on a self-hosted CodeBuild runner instead of GitHub-hosted runners. Changes
ImpactWorkflow execution for the migration tests now targets self-hosted CodeBuild infrastructure rather than GitHub-hosted runners, affecting where the jobs execute (e.g., resource and environment provider). WalkthroughThe migration-tests workflow was modified to change the execution environment for three jobs: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
.github/workflows/migration-tests.yaml (2)
104-104: Verify matrix-job concurrency semantics for the shared runner label.The
prepare-and-migratejob uses a 4-entry matrix (mysql, mssql, postgre, oracle) but all four instances will request the identical runner labelcodebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }}(the label does not includematrix.db_typeorstrategy.job-index). AWS CodeBuild's GitHub Actions integration is designed to start one CodeBuild build per queued job with that label, so this typically works, but please confirm in a test run that all four matrix legs actually get concurrent runners (and aren't serialized or collapsed onto one). If you observe queuing, append a matrix-unique suffix to the label, e.g.:♻️ Optional: make the label unique per matrix leg
- runs-on: codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }} + runs-on: codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }}-${{ strategy.job-index }}Additionally, several steps assume a GitHub-hosted Ubuntu runner environment:
- Line 415–416:
sudo apt-get update && sudo apt-get install -y default-mysql-client python3 python3-pip— requires Debian/Ubuntu with passwordlesssudoand network access to apt repos.- Line 418–419: the comment "Docker is already available in GitHub Actions runners" is now stale; please update it or confirm the CodeBuild image ships with Docker daemon access (not just the CLI) so
docker run -dwith port bindings onhost.docker.internalworks.- Line 804:
python3 -m http.server 8000needs port 8000 free on the runner host and routable from inside Docker viahost.docker.internal:host-gateway.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/migration-tests.yaml at line 104, The prepare-and-migrate job's runner label is identical across matrix legs causing possible serialization; update the runs-on label string (the literal "codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }}") to append a matrix-unique suffix such as "${{ matrix.db_type }}" or "${{ strategy.job-index }}", run a test workflow to confirm all four matrix legs get concurrent CodeBuild builds, and if queuing is observed keep the unique suffix. Also verify the CodeBuild image used by this job actually provides sudo+apt, Docker daemon access (not just the CLI), and allows host-gateway host.docker.internal port binding — either change the image to one that does or adjust the steps that assume Ubuntu/apt and docker run; finally ensure the python3 -m http.server 8000 step can bind and be reachable from containers (or change to an alternate networking approach or free port) and update or remove the stale comment about Docker availability accordingly.
1519-1519: LGTM onversion-bumprunner change, but double-check git push permissions on the CodeBuild runner.The
version-bumpjob writes back to the repo (lines 1549–1556) usinggithub-actions[bot]andgit push origin HEAD:${GITHUB_REF#refs/heads/}. On GitHub-hosted runners theGITHUB_TOKENprovided byactions/checkout@v4is auto-configured as the credential helper; AWS CodeBuild-managed GitHub Actions runners also propagateGITHUB_TOKENwith the declaredcontents: writepermission, so this should continue to work. Please verify on the first run that the push actually succeeds (no403from the push step) and that branch protection rules onmasterallow the bot push — the runner swap doesn't change those constraints but is a good moment to re-check.Also note: this job runs on schedule (nightly cron, line 7) and, after the runner change, every nightly run will still spin up a CodeBuild build even when
needs.update-and-build.outputs.latest_versionequals the current version (theif:guard is only on the single push step at line 1539, not the job). Consider adding a job-levelif:to avoid paying for an idle CodeBuild minute, e.g.:♻️ Optional: skip the job when there is nothing to bump
version-bump: needs: [update-and-build, prepare-and-migrate] runs-on: codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }} + if: needs.update-and-build.outputs.latest_version != '' && needs.update-and-build.outputs.latest_version != needs.update-and-build.outputs.current_version permissions: contents: write🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/migration-tests.yaml at line 1519, The version-bump job now runs on a CodeBuild runner (runs-on: codebuild-wso2_product-apim-...), so verify the push step that uses git push origin HEAD:${GITHUB_REF#refs/heads/} actually succeeds with the GITHUB_TOKEN credential provided by actions/checkout@v4 on the new runner and that branch protection allows github-actions[bot] pushes; run the pipeline once and confirm there is no 403 on the push step and that the commit appears in the target branch. Additionally, to avoid wasting CodeBuild minutes, add a job-level if: condition to the version-bump job (using the same needs.update-and-build.outputs.latest_version check used for the push step) so the entire CodeBuild job is skipped when there is nothing to bump.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/migration-tests.yaml:
- Line 11: Update the runner label and environment verification: confirm an AWS
CodeBuild project named wso2_product-apim exists and is configured for GitHub
Actions webhook integration and that its compute image includes Docker, Java 21,
Python 3, unzip, wget, zip and sudo/apt-get; then modify the runs-on label
(currently "runs-on: codebuild-wso2_product-apim-${{ github.run_id }}-${{
github.run_attempt }}") to include the matrix context (e.g. append ${{ matrix.db
}} or ${{ matrix.database }}) so concurrent matrix jobs (the matrix job that
spawns MySQL/MSSQL/Postgres/Oracle) request distinct runner labels and avoid
runner contention.
---
Nitpick comments:
In @.github/workflows/migration-tests.yaml:
- Line 104: The prepare-and-migrate job's runner label is identical across
matrix legs causing possible serialization; update the runs-on label string (the
literal "codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt
}}") to append a matrix-unique suffix such as "${{ matrix.db_type }}" or "${{
strategy.job-index }}", run a test workflow to confirm all four matrix legs get
concurrent CodeBuild builds, and if queuing is observed keep the unique suffix.
Also verify the CodeBuild image used by this job actually provides sudo+apt,
Docker daemon access (not just the CLI), and allows host-gateway
host.docker.internal port binding — either change the image to one that does or
adjust the steps that assume Ubuntu/apt and docker run; finally ensure the
python3 -m http.server 8000 step can bind and be reachable from containers (or
change to an alternate networking approach or free port) and update or remove
the stale comment about Docker availability accordingly.
- Line 1519: The version-bump job now runs on a CodeBuild runner (runs-on:
codebuild-wso2_product-apim-...), so verify the push step that uses git push
origin HEAD:${GITHUB_REF#refs/heads/} actually succeeds with the GITHUB_TOKEN
credential provided by actions/checkout@v4 on the new runner and that branch
protection allows github-actions[bot] pushes; run the pipeline once and confirm
there is no 403 on the push step and that the commit appears in the target
branch. Additionally, to avoid wasting CodeBuild minutes, add a job-level if:
condition to the version-bump job (using the same
needs.update-and-build.outputs.latest_version check used for the push step) so
the entire CodeBuild job is skipped when there is nothing to bump.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 550a068b-10c2-4ff2-ae2b-a77a3d1c8a13
📒 Files selected for processing (1)
.github/workflows/migration-tests.yaml
| jobs: | ||
| update-and-build: | ||
| runs-on: ubuntu-latest | ||
| runs-on: codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }} |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
AWS CodeBuild GitHub Actions managed self-hosted runner label format runs-on codebuild project name run_id run_attempt
💡 Result:
The label format for AWS CodeBuild's managed self-hosted GitHub Actions runner, used in the runs-on key of GitHub Actions workflows, is codebuild--${{ github.run_id }}-${{ github.run_attempt }}. This is the required base label, where is replaced with the exact name of your CodeBuild project configured with the GitHub webhook. The ${{ github.run_id }} and ${{ github.run_attempt }} are GitHub context variables that uniquely identify the workflow run and attempt, ensuring proper mapping and cleanup. Example: jobs: build: runs-on: codebuild-myProject-${{ github.run_id }}-${{ github.run_attempt }} steps: - uses: actions/checkout@v4 Optional overrides can be added as additional labels or legacy suffixes, such as: - Multi-label: image:arm-3.0, instance-size:medium, fleet:myFleet, buildspec-override:true - Legacy single label: codebuild--${{ github.run_id }}-${{ github.run_attempt }}-image-arm-3.0-instance-size-medium-with-buildspec This format enables CodeBuild to process GitHub Actions jobs on managed self-hosted runners with native AWS integration.
Citations:
- 1: https://docs.aws.amazon.com/codebuild/latest/userguide/sample-github-action-runners-update-labels.html
- 2: https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner-overview.html
- 3: https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner-questions.html
- 4: https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html
- 5: https://aws.amazon.com/blogs/devops/aws-codebuild-managed-self-hosted-github-action-runners/
🏁 Script executed:
cat -n .github/workflows/migration-tests.yaml | head -120Repository: wso2/product-apim
Length of output: 5182
🏁 Script executed:
cat -n .github/workflows/migration-tests.yaml | sed -n '101,250p'Repository: wso2/product-apim
Length of output: 8448
🏁 Script executed:
cat -n .github/workflows/migration-tests.yaml | sed -n '310,330p'Repository: wso2/product-apim
Length of output: 1092
🏁 Script executed:
# Check for tool usage in the workflow
rg -n "(docker|java|python|unzip|wget|zip|apt-get|apt)" .github/workflows/migration-tests.yaml | head -40Repository: wso2/product-apim
Length of output: 4137
🏁 Script executed:
# Check if Python or apt-get/sudo are used in the workflow
rg -n "(python|apt-get|apt |sudo)" .github/workflows/migration-tests.yamlRepository: wso2/product-apim
Length of output: 596
🏁 Script executed:
# Check if zip command is actually used (not just unzip)
rg -n "^\s+zip " .github/workflows/migration-tests.yamlRepository: wso2/product-apim
Length of output: 110
Runner label format is correct; verify CodeBuild project configuration and compute image capabilities.
The label codebuild-wso2_product-apim-${{ github.run_id }}-${{ github.run_attempt }} follows the AWS CodeBuild documented pattern. However, confirm that an AWS CodeBuild project named wso2_product-apim exists in your target account/region and is configured with GitHub Actions webhook integration; otherwise jobs will hang indefinitely waiting for a runner.
Also verify that the project's compute image includes Docker, Java 21, Python 3, unzip, wget, zip, and sudo-capable apt-get, as these tools are required by the workflow steps (lines 314–316, 332–341, 401, 415–416, 804, 1384).
Note: The matrix job (line 104) spawns four concurrent database variants (MySQL, MSSQL, PostgreSQL, Oracle) all requesting the same runner label. Ensure your CodeBuild configuration can provision multiple runner instances concurrently for the same label, or consider adding matrix context to the label to differentiate them.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/migration-tests.yaml at line 11, Update the runner label
and environment verification: confirm an AWS CodeBuild project named
wso2_product-apim exists and is configured for GitHub Actions webhook
integration and that its compute image includes Docker, Java 21, Python 3,
unzip, wget, zip and sudo/apt-get; then modify the runs-on label (currently
"runs-on: codebuild-wso2_product-apim-${{ github.run_id }}-${{
github.run_attempt }}") to include the matrix context (e.g. append ${{ matrix.db
}} or ${{ matrix.database }}) so concurrent matrix jobs (the matrix job that
spawns MySQL/MSSQL/Postgres/Oracle) request distinct runner labels and avoid
runner contention.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #14175 +/- ##
============================================
- Coverage 19.05% 9.15% -9.90%
+ Complexity 1419 718 -701
============================================
Files 361 361
Lines 17719 17719
Branches 1897 1897
============================================
- Hits 3377 1623 -1754
- Misses 14301 16070 +1769
+ Partials 41 26 -15
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
VenukshiMendis
force-pushed
the
master-add-oracle-support
branch
from
26edaf2 to
57fa133
Compare
1 participant
$subject