actions: debootstrap: Add parent-suite property to describe base suite #424

obbardc · 2023-08-04T08:21:43Z

No description provided.

sjoerdsimons · 2023-08-04T08:27:09Z

Would be great if you could add a test for this case

obbardc · 2023-08-04T16:19:23Z

Would be great if you could add a test for this case

as discussed, next step is to add some CI tests to build old/new debian/ubuntu/kali releases :-)

obbardc · 2023-08-10T10:06:53Z

cc @elboulangero for the Kali rolling tests. Is that good enough ?

elboulangero · 2023-08-15T09:09:41Z

cc @elboulangero for the Kali rolling tests. Is that good enough ?

It's excellent, thanks very much for this work. Tested it, works as expected for Kali.

tests/kali/test.yaml

sjoerdsimons

lots of questins; need some pondering if this is the right shape

sjoerdsimons · 2024-01-13T15:41:16Z

actions/debootstrap_action.go

-	cmdline = append(cmdline, "/usr/share/debootstrap/scripts/unstable")
+
+	if len(d.Script) > 0 {
+		if _, err := os.Stat(d.Script); err != nil {


this should be done in verify not at run time

.github/workflows/ci.yaml

sjoerdsimons · 2024-01-13T15:49:36Z

.github/workflows/ci.yaml

+          - { name: "debian (bookworm armhf)", case: "debian", variables: "-t architecture:armhf -t suite:bookworm" }
+          - { name: "debian (trixie amd64)", case: "debian", variables: "-t architecture:amd64 -t suite:trixie" }
+          - { name: "debian (trixie arm64)", case: "debian", variables: "-t architecture:arm64 -t suite:trixie" }
+          - { name: "debian (trixie armhf)", case: "debian", variables: "-t architecture:armhf -t suite:trixie" }


this massively extends the number of test jobs; Does testing trixie on arm64/armhf give us extra data? Also i'm a tad wary about it breaking CI non-deterministically

as was done to tune our runs; for trixie for now please just build amd64 and only on kvm

yeah probably best to remove trixie

I removed the CI jobs completely. Not sure they add much value really.

sjoerdsimons

So spending a bunch of time today thinking about it and remembering what the issue is again for background

Fundamentally the main part of this problem happens because the newer debootstrap scripts for Debian forced installing usr-is-merged (and to not install usrmerge), unless it detected a known "old" debian suite. Note that this didn't generally impact e.g. Ubuntu, tanglu or trisquel (as their debootstrap scripts don't do that).

Fundamentally the fix we did in debos was to exclude usr-is-merged unless we know the bootstrap distro was a newer debian to not break existing image builds for stable debian releases. For extra fun this never impacted apertis v2024 (bookworm based) as its init-system-helpers hard depend on usr-is-merged and the --exclude switch only applies to the initial package selection not on the expanded dependency list (cause usr-is-merged to be installed on bootstrap in apertis even though it was excluded). On Debian >= bookworm and some derivatives init-system-helpers depends on usrmerge | usr-is-merged , which with the exclusion by both debos and the debootstrap scripts meant neither got installed hence the issues :).

However it should be noted that this kind of gymnastics from debootstrap is exceedingly rare as should be debos doing these workaround.

Slightly orthogonally debos always forced the Debian debootstrap scripts (unstable specifically but they're all the same for any recentish debian release). Even though e.g. kali, ubuntu, tanglu, etc do have their own custom (to a smaller or bigger extend) debootstrap scripts.

So overall i think the step forward for debos should be to be able to use more appropriate debootstrap scripts (e.g. the ubuntu ones for ubuntu); e.g. either matched on the suite (if it's has it's own script directly) or by a property that indicates which suite to use by debootstrap (usually the direct parent, but there could be a longer lineage) or for more custom handling a deboostrap script as part of the recipe (for those needing a custom scripts, but don't have it available in debootstrap).

Most of that matches the patches you've already done; Though to paint some bikesheds:

Change parent-suite into debootstrap-suite instead, with the semantic that the configured argument is what "suite" to use for debootstrap purposes (which primarily selects the right "builtin" debootstrap script but optionally does workarounds).
Change script into debootstrap-script with the argument being the path to the debootstrap script inside the recipe origin rather then a full path.

Having both a debootstrap-script and debootstrap-suite should cause an error as it makes no real sense. If you use a script and need workarounds they should be done in the custom script instead at which point the debootstrap-suite setting has no meaning anymore :)

Also at the vary least if the suite has a builtin debootstrap script then having a debootstrap-suite should at the very least be a warning but i'd even error on the side of it being an error as it's both confusing and a very weird situation.

Having a debootstrap-script in that situation should not be as problematic as e.g. someone is building a variant not supported in the debootstrap scripts. But the output should clearly indicate a custom script is used (probably in the debootstrap output)

For the current behaviour; we should use the current default "unstable" script as always (but also indicate the fallback in the output and potentially suggest to user to set debootstrap-suite) if suite doesn't have it's own script and neither of the new properties were given. In other words automatic fallback only happens if the user didn't explicitly ask for a specific suite or script for debootstrapping

For the usr-is-merged workaround we should only apply it when we detect a known old distribution (based on the suite used debootstrapping) but assume anything not specifically recognized doesn't need this workaround. This is similar to how debootstrap does this (though it only goes from the remote codename rather then the script name) and will be more future proof.

Your current patchset already gets quite closed to this so the changes should be minimal. For e.g. kali that means there is no reason to set a debootstrap-suite as it already has its scripts in debootstrap and we'll pick those up now. For apertis v2024 the same goes (though building will correctly suggest it should be for consistency), while apertis v2023 (and older) now need a debootstrap-suite setting to build image on a newer debian base/debos, but I think that's fine.

sjoerdsimons · 2024-01-21T09:01:29Z

actions/debootstrap_action.go

-func (d *DebootstrapAction) isLikelyOldSuite() bool {
-	switch strings.ToLower(d.Suite) {
+// Check if suite is something before usr-is-merged was introduced
+func shouldExcludeUsrIsMerged(suite string) bool {


We should flip this function around so the default is false as it won't be needed for newer suites (whose set will increase) but will be for older debian ones (whose set won't change).. It also should default to false for newer debian derivatives (using >= bookworm) .

fwiw debootstrap has the following list: etch*|lenny|squeeze|wheezy|jessie*|stretch|buster|bullseye

no need to deal with jessie* though given it is completely unsupported since 2020

sjoerdsimons · 2024-01-21T09:29:31Z

actions/debootstrap_action.go

+- parent-suite -- release code name which this suite is based on. Useful for downstreams which do
+  not use debian codenames for their suite names (e.g. "stable").


Fwiw this could be used for non-debian parents as well.. E.g. parent-suite could be jammy for ubuntu derivatives (or kali for kali derivatives). We should require the parent-suite to be known by debootstrap (as in a script exists) if given. (potentially this requirement should only exist if there is no script for suite, but in that case this property is useless).

sjoerdsimons · 2024-01-21T13:19:58Z

.github/workflows/ci.yaml

+          - { name: "debian (bookworm armhf)", case: "debian", variables: "-t architecture:armhf -t suite:bookworm" }
+          - { name: "debian (trixie amd64)", case: "debian", variables: "-t architecture:amd64 -t suite:trixie" }
+          - { name: "debian (trixie arm64)", case: "debian", variables: "-t architecture:arm64 -t suite:trixie" }
+          - { name: "debian (trixie armhf)", case: "debian", variables: "-t architecture:armhf -t suite:trixie" }


as was done to tune our runs; for trixie for now please just build amd64 and only on kvm

obbardc · 2025-08-11T14:29:54Z

Looks like I messed up the rebase; took some time and sorted it out. I will look at the remaining comments next.

obbardc · 2025-11-30T23:33:17Z

I rebased on main; fixed lint issues and removed the CI tests (for now). Will look at solving review comments and best way to add CI tests next.

obbardc · 2025-12-08T01:28:53Z

@sjoerdsimons I squashed the commits and updated the commit message to hopefully be a bit more clear. Let me know if you want any changes.

Introduce a new `parent-suite` property for the debootstrap action to describe the upstream Debian suite a downstream is based on. If unset it defaults to the `suite` value. This property is then used to decide whether to apply the `usr-is-merged` workaround based on a fixed list of older Debian suites, defaulting to no workaround for newer and unknown suites and derivatives. Add `debootstrap-suite` and `debootstrap-script` properties to give more control over which debootstrap script is used: - `debootstrap-suite` selects a builtin debootstrap script for the given suite and is intended for downstreams whose own suite name does not have a builtin script. - `debootstrap-script` points to a custom script inside the recipe origin and is cleaned and verified at parse time. Only one of `debootstrap-suite` or `debootstrap-script` may be set. When no explicit script is provided, automatically choose a script by trying (in order) the `suite`, the `parent-suite` and finally `unstable`, logging what script is used. The `usr-is-merged` workaround is applied only for suites which are known to need it. Existing recipes that do not use the new properties keep their previous behaviour, while downstream distributions gain a more precise way to describe their relationship to Debian and the debootstrap scripts which they depend on. Fixes: #361 Signed-off-by: Christopher Obbard <[email protected]>

obbardc requested a review from sjoerdsimons August 4, 2023 08:21

obbardc self-assigned this Aug 4, 2023

obbardc mentioned this pull request Aug 4, 2023

New debootstrap is unable to bootstrap pre-bookworm debian derivatives #361

Open

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 38c4eb0 to 2bab6e7 Compare August 4, 2023 09:46

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 2bab6e7 to 34532f0 Compare August 10, 2023 10:05

obbardc changed the title ~~actions: debootstrap: Add property parent-suite to indicate which suite bootstrapping should be done for~~ actions: debootstrap: Add parent-suite property to indicate which suite a downstream is based on Aug 10, 2023

elboulangero reviewed Aug 15, 2023

View reviewed changes

tests/kali/test.yaml Outdated Show resolved Hide resolved

elboulangero reviewed Aug 15, 2023

View reviewed changes

tests/kali/test.yaml Outdated Show resolved Hide resolved

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 34532f0 to 99fcb9e Compare August 15, 2023 09:33

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 99fcb9e to 8eecdbb Compare January 10, 2024 14:35

obbardc added this to the v1.1.4 milestone Jan 10, 2024

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch 3 times, most recently from 7b53bef to 7b6ce52 Compare January 10, 2024 16:46

sjoerdsimons reviewed Jan 13, 2024

View reviewed changes

sjoerdsimons linked an issue Jan 21, 2024 that may be closed by this pull request

New debootstrap is unable to bootstrap pre-bookworm debian derivatives #361

Open

sjoerdsimons requested changes Jan 21, 2024

View reviewed changes

sjoerdsimons mentioned this pull request Feb 24, 2025

actions: Add mmdebstrap action #515

Merged

obbardc moved this from Review required to Stalled in Debos&Fakemachine development Aug 8, 2025

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch 3 times, most recently from e75a972 to 6698804 Compare August 11, 2025 14:28

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch 2 times, most recently from f00a529 to 5bccaa9 Compare November 30, 2025 23:31

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch 2 times, most recently from ae1e5ca to 53aa801 Compare December 8, 2025 01:27

obbardc changed the title ~~actions: debootstrap: Add parent-suite property to indicate which suite a downstream is based on~~ actions: debootstrap: Refine automatic debootstrap script selection Dec 8, 2025

obbardc requested a review from sjoerdsimons December 8, 2025 01:29

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 53aa801 to f4b89c9 Compare December 8, 2025 01:33

obbardc changed the title ~~actions: debootstrap: Refine automatic debootstrap script selection~~ actions: debootstrap: Add parent-suite property to describe base suite Dec 8, 2025

		- parent-suite -- release code name which this suite is based on. Useful for downstreams which do
		not use debian codenames for their suite names (e.g. "stable").

actions: debootstrap: Add parent-suite property to describe base suite #424

Are you sure you want to change the base?

actions: debootstrap: Add parent-suite property to describe base suite #424

Uh oh!

Conversation

obbardc commented Aug 4, 2023

Uh oh!

sjoerdsimons commented Aug 4, 2023

Uh oh!

obbardc commented Aug 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

obbardc commented Aug 10, 2023

Uh oh!

elboulangero commented Aug 15, 2023

Uh oh!

Uh oh!

Uh oh!

sjoerdsimons left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

obbardc commented Aug 11, 2025

Uh oh!

obbardc commented Nov 30, 2025

Uh oh!

obbardc commented Dec 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

obbardc commented Aug 4, 2023 •

edited

Loading