feat(xcat-dep): add the Ubuntu/Debian sbuild dependency-build matrix - #63
Open
dhilst wants to merge 4 commits into
Open
feat(xcat-dep): add the Ubuntu/Debian sbuild dependency-build matrix#63dhilst wants to merge 4 commits into
dhilst wants to merge 4 commits into
Conversation
The Ubuntu/Debian .deb dependency build lived only in the xCAT CI
(xcat-core-ci-cd), so xcat-dep could not build its apt dependency packages
standalone. Bring the two CI-only drivers into the repo, alongside the existing
build-apt-repo.sh + per-package make_deb.sh:
- mk-dep-chroots.sh : create the per-codename sbuild chroots (root, per host).
- build-dep-debs.sh : build every dep .deb per codename INSIDE the matching
sbuild chroot (correct libc/toolchain) and stage them for
build-apt-repo.sh; xCAT-genesis-base (Arch:all) built once
and staged into every codename (cross-arch netboot, #7610).
Document the Ubuntu/Debian sbuild flow in BUILD.md. The scripts still carry
CI-specific path assumptions (a shared-tree bind-mount in mk-dep-chroots.sh);
genericizing them is tracked separately, and the CI keeps its own copies until
this lands upstream.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
…all.pl) The Ubuntu/Debian dependency build shipped as three bash scripts (build-dep-debs.sh, build-apt-repo.sh, mk-dep-chroots.sh) whose review (PR xcat2#63) surfaced correctness problems: partial/stale output could be published, the rpm->deb genesis conversion dropped the maintained package semantics (Depends/Breaks/Replaces + maintainer scripts), the arch matrix was invalid (x86-only syslinux/elilo/xnba treated as ppc64el packages, and Architecture:all packages with no single producer), several required failures exited zero, and the build/repo scripts disagreed on their staging path and codename set (focal missing from the assembler). Rewrite it as proper, unit-tested Perl mirroring the EL side (mockbuild-all.pl / MockBuildUtils.pm / <dep>/mockbuild.pl / t/*.t / packages-manifest.conf), sharing one CLI vocabulary: - BuildUtils.pm: shared, testable helpers + the canonical CLI spec, plus the Debian-specific helpers (out-of-tree changelog stamping, genesis control preservation, deb inspection, cross-arch genesis provisioning). - sbuild-all.pl: the orchestrator, absorbing all three shell scripts. Builds + validates into a fresh per-arch staging tree and only (re)assembles the published apt repo from validated staging -- so partial/failed output never ships and stale debs never accumulate. Auto-initializes the per-codename sbuild chroots on first run. Fails the whole run non-zero on any missing chroot/package/artifact or version-pin mismatch. - <dep>/sbuild.pl x7: per-package builders that drive each package's MAINTAINED debian/ in the matching chroot (never re-implemented), so the converted/built packages keep their control metadata and maintainer scripts. - debs-manifest.conf: per-[<codename>-<arch>] required set + version pins, encoding the per-arch package sets (x86 boot components built once on amd64 as the single producer; ppc64el builds only the arch-specific compiled deps). - t/sbuild-all.t: fixture tests for every pure helper. - goconserver/make_deb.sh: pin the upstream SHA instead of cloning a moving branch, so every matrix cell builds the same source (reproducible). Codename set unified across build, assembly, chroots and docs (focal IS supported). BUILD.md documents the new flow. Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
…oot build Each <dep>/sbuild.pl copies the built .deb(s) to --result-dir from INSIDE the schroot session. That only reaches the host when --result-dir is on a path bind-mounted into the chroot (the shared /opt/xcat-ci-shared tree, as the CI uses). A mis-configured result-dir (e.g. a chroot-local /tmp) would let the build "succeed" yet leave nothing on the host -- a silent no-output. Verify host-side after the session that the .deb(s) are actually present and fail loud with a pointed message otherwise (consistent with the fail-hard design). Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The pipeline invokes it as ./sbuild-all.pl; without the exec bit that returns 126 (Permission denied). Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Ubuntu/Debian .deb dependency build lived only in the xCAT CI (xcat-core-ci-cd), so xcat-dep could not build its apt dependency packages standalone. Bring the two CI-only drivers into the repo, alongside the existing build-apt-repo.sh + per-package make_deb.sh:
Document the Ubuntu/Debian sbuild flow in BUILD.md. The scripts still carry CI-specific path assumptions (a shared-tree bind-mount in mk-dep-chroots.sh); genericizing them is tracked separately, and the CI keeps its own copies until this lands upstream.