Forward-port: Docker- and host-based CI from openifs-lts/CY48R1.1#7
Draft
Adehill wants to merge 1 commit into
Draft
Forward-port: Docker- and host-based CI from openifs-lts/CY48R1.1#7Adehill wants to merge 1 commit into
Adehill wants to merge 1 commit into
Conversation
* Add Docker build, Docker CI driver, and GitHub Actions workflow
Introduce three new automation entry points for OpenIFS, plus a shared
helper library:
- scripts/bootstrap/docker/ — container-based OpenIFS install: builds
the toolchain image (apt + OpenMPI), clones or copies the source, and
runs the test suite end-to-end. Replaces the legacy scripts/docker/
- scripts/ci/docker_ci/ — branch-vs-branch SAVED_NORMS bit-compare
driver: builds a control branch and a test branch in matching images,
runs openifs-test.sh -cbt, and writes a self-contained text report
- scripts/shared/ — shared library used by both drivers
(shared_helpers, docker_lib, ci_lib, read_yml_config, setup_logging,
find_py_packages)
- .github/workflows/bit-compare-docker.yml — GitHub Actions workflow
that drives the docker_ci bit-compare on push/PR
Removes the legacy scripts/docker/ tree (superseded by bootstrap/docker)
and updates README.md, .gitignore, and oifs-config.edit_me.sh to point
at the new paths.
* Drop hardcoded fork URL from CI workflow and script defaults
The workflow and the two YAML defaults all hardcoded the personal
fork `Adehill/openifs.git`, which would break the workflow when run
in any other repo and present a confusing default to anyone running
the bootstrap script locally.
- .github/workflows/bit-compare-docker.yml: replace the hardcoded
URL in `Resolve control SHA` and in the rendered `ci_test_docker.yml`
with `${{ github.repository }}`, so the workflow follows whichever
fork it lives in (Adehill/openifs for testing,
ecmwf-ifs/openifs upstream)
- scripts/bootstrap/docker/config/create_openifs_docker.yml: default
`openifs_repo_url` to `https://github.com/ecmwf-ifs/openifs.git`
- scripts/ci/docker_ci/config/ci_test_docker.yml: same default change
(this file is only read for local script runs — the GitHub Actions
workflow renders its own copy in /tmp)
* Simplify the workflow yaml so that it does not include defaults that are not changed in the CI
* Changed the workflow to match the template of ecland and ecrad
* Derive cycle from bundle.yml and use LTS branch as bit-compare control
Find the cycle version in bundle.yml so the workflow no
hardcoded 48r1, then use the matching openifs-lts/CY* branch as the push
control (falling back to main if it does not yet exist), and skip
direct pushes to LTS branches since changes land there via PR. Important to note this needs to be updated to allow unsupported branches, when they are defined
* Trigger CI only on main pushes and harden env var handling
Narrow the push trigger to main so feature-branch pushes no longer
double-trigger alongside their pull request, and pass the PR base ref
through a typed env: block rather than ${{ }} interpolation inside the
run script to remove a shell-injection sink.
* Fix shutil.copytree recursion when build dir is inside test source
Move WORK_DIR out of github.workspace in the CI workflow so the test
phase no longer copies the workspace into a subdirectory of itself,
and add _oifs_docker_ci to populate_from_local's ignore patterns so
the helper is safe against the same nested layout elsewhere. This better replicates the local docker_ci workflow, while preventing a fail on the github action CI workflow
* Add host-based bit-compare CI and extend docker matrix to gcc 12/13/14
Add ci-oifs-host.py driver and a bit-compare-host.yml workflow that
stage control and test sources, build and test on the runner directly
(no Docker), and bit-compare SAVED_NORMS. The driver reuses the same
shared helpers and ci_lib as ci-oifs-docker.py so report shape and CI
semantics match. Also expand the docker workflow to run gcc 12, 13,
and 14 in parallel.
* Rebalance gcc matrix: host carries 12/13/14, docker keeps gcc-14 only
Move the gcc breadth to the host workflow so compiler coverage runs on
the install path most users follow (apt + bundle build on Ubuntu).
Docker stays as a single gcc-14 reproducibility anchor with pinned
OpenMPI 5.0.10, leaving four parallel jobs per PR instead of six.
* Capture build output and ensure test-phase failures still write a report
Tee the configure+build stage to its own file (in addition to the
ctest tee that already existed), enable pipefail so neither stage's
exit code is masked by tee, and append both BUILD OUTPUT and CTEST
OUTPUT sections to the bit-compare report. Wrap the test phase in
both drivers so a build or ctest failure writes a synthetic report
with the captured outputs instead of crashing main and leaving the
artifact without a top-level report.
* Capture ctest LastTest.log so per-test failure details is reported. Also, change the default docker version to 13.
* fix typo and re-test
* Improve CI summary branch status reporting
Report build and test status for both control and test branches, including
the passing test branch when control fails. Align CI summary fields, include
bit-comparison skip reasons, and keep verbose build/ctest logs as separate
artifacts instead of embedding them in the summary report.
* Show test branch name (and short SHA) in CI summary
When `test_branch` is a local checkout (the github.workspace path in
GitHub Actions), the summary previously printed the raw path. Resolve
it to "branch (sha7)" via git, falling back to GITHUB_HEAD_REF /
GITHUB_REF_NAME when actions/checkout leaves HEAD detached.
* Potential fix for pull request finding - handle a branch/ref that can not be resolved in _resolve_control_sha
_resolve_control_sha() assumes git ls-remote returns a non-empty line and blindly indexes out.split()[0]. If the branch/ref can’t be resolved (typo, missing branch, auth issue), this will raise an IndexError and obscure the real problem. Handle empty output explicitly and raise a clear exception (or return a sentinel) with the branch/ref that failed to resolve.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit de869db)
msleigh
force-pushed
the
main
branch
2 times, most recently
from
7e2559f to
7b8c12e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picks the CI changes merged to openifs-lts/CY48R1.1, including the follow-up that disables host CI on pull requests so only Docker CI runs on PRs. The host workflow remains available on pushes to main and via workflow_dispatch.