Add `install alongside`

[cgwalters]

Right now we have

podman run --rm --privileged --pid=host -v /:/target --security-opt label=type:unconfined_t "${image}" bootc install-to-filesystem --karg=console=ttyS0,115200n8 --replace=alongside /target

We can first consider simplifying like this by adding a new toplevel to-existing-root:

podman run --rm --privileged --pid=host --security-opt label=type:unconfined_t -v /:/target "${image}" bootc install to-existing-root

or so, which would assume installing to /target. This consolidates --replace=alongside and /target to just to-existing-root.

I don't see a reason to support installing alongside to not the rootfs.

Now, if we can switch to using dynamic mounts then we can also drop the -v /:/target - I'd like to test this out as it could simplify a lot of our code.

Handling kargs in alongside

When doing an alongside install, there are a ton of use cases for picking up things from the existing environment:

• root ssh keys
• console= kargs

Or in general, picking up "configuration".

I am a bit uncertain if we try to add a lot of sugar to bootc in the near term for this versus having higher level tools do it, but...

We could in to-existing-root also have something like --import-kargs=console=* or so to fetch the existing console kernel args (match by glob).

[cgwalters]

Bigger picture though, while "alongside" installs were easy to implement (it's just small bits of code on top of to-filesystem I do think what we really want to encourage people to do is takeover installs because that ends up fixing several things:

• We don't "leak" the entire existing prior root
• We can switch filesystem types (including supporting initializing LUKS in the cloud, etc.)

[cgwalters]

This all said I think in practice people should be automating these invocations, not typing them by hand. It's already very doable by scripting in e.g. cloud-init and that's the intended design.

[rhatdan]

I agree on takeover installs, this is what most people will want, since I don't believe there is any easy way back to the previous root OS. Being able to maintain /home would be nice, but in most cases we expect people to just boot an existing image and convert it to an Bootable OCI Image.

[rhatdan]

I read about dynamic mounts but not sure how this can allow access to / from within the privileged container without Podman requiring a change?

[cgwalters]

I read about dynamic mounts but not sure how this can allow access to / from within the privileged container without Podman requiring a change?

I believe we can get access to the root mount namespace at least because we have --pid=host so we inherently have /proc/1/ns/mnt. (Also in practice we're running with --privileged so we can do anything - we could inject a kernel module that does whatever we want, etc. But this is about doing it in a sane/supportable way)

[rhatdan]

Ok, good idea.

[cgwalters]

One part in #382

[cgwalters]

One thing I wanted to sanity check here is; do we have open_tree in current c9s? And we appear to, as this works

diff --git a/lib/Cargo.toml b/lib/Cargo.toml
index 6491bc3..bd5ccc4 100644
--- a/lib/Cargo.toml
+++ b/lib/Cargo.toml
@@ -32,7 +32,7 @@ openssl = "^0.10.64"
 # TODO drop this in favor of rustix
 nix = { version = "0.28", features = ["ioctl", "sched"] }
 regex = "1.10.3"
-rustix = { "version" = "0.38", features = ["thread", "fs", "system", "process"] }
+rustix = { "version" = "0.38", features = ["thread", "fs", "system", "process", "mount"] }
 schemars = { version = "0.8.16", features = ["chrono"] }
 serde = { features = ["derive"], version = "1.0.197" }
 serde_ignored = "0.1.10"
diff --git a/lib/src/install.rs b/lib/src/install.rs
index c017821..38f7c19 100644
--- a/lib/src/install.rs
+++ b/lib/src/install.rs
@@ -1204,6 +1204,7 @@ fn clean_boot_directories(rootfs: &Dir) -> Result<()> {
 /// Implementation of the `bootc install to-filsystem` CLI command.
 #[context("Installing to filesystem")]
 pub(crate) async fn install_to_filesystem(opts: InstallToFilesystemOpts) -> Result<()> {
+    use rustix::mount::OpenTreeFlags;
     let fsopts = opts.filesystem_opts;
     let root_path = &fsopts.root_path;
 
@@ -1219,6 +1220,14 @@ pub(crate) async fn install_to_filesystem(opts: InstallToFilesystemOpts) -> Resu
         anyhow::bail!("Not a root mountpoint: {root_path}");
     }
 
+    let host_root = rustix::mount::open_tree(
+        rootfs_fd.as_fd(),
+        "/proc/1/ns/mnt",
+        OpenTreeFlags::OPEN_TREE_CLOEXEC,
+    )
+    .context("Opening host root")?;
+    dbg!(host_root);
+
     // Gather global state, destructuring the provided options
     let state = prepare_install(opts.config_opts, opts.source_opts, opts.target_opts).await?;
 

Also for my own reference, the man pages are over here https://github.com/brauner/man-pages-md (because groff stinks apparently, which I agree with).