FreeBSD support

[bsdimp]

FreeBSD now has loader.kboot for booting from Linux into FreeBSD. this works really well on aarch64, and will soon work well for amd64 (once I find the time to finish it).

I've been asked if ZBM could support FreeBSD booting. Looking quickly at the code, it looks like that could be accommodated with a custom detection for FreeBSD and a different boot recipe using loader.kboot instead of the kexec tools that the current script uses.

Would you be open to working together to make this happen?

[zdykstra]

FreeBSD booting has been a long-standing project "want" - we'd absolutely welcome it! Roughly speaking, how would the loader.kboot process work?

[ahesford]

+1 for this. I've dreamt of FreeBSD support ever since the OpenZFS unification.

[bsdimp]

To boot entirely off a ZFS BE, the process would be approximately

loader.kboot bootdev=zfs:
or
loader.kboot bootdev=zfs:/zroot/ROOT/gerbils

Setting 'bootdev' like this will cause the boot loader to use that as the 'load' device for both the loader's file (lua scripts, loader.conf) as well as finding the kernel here. Both will also clear 'bootonce' if it is set, but the latter will boot the geribils BE regardless of its current bootonce setting. For this project, the second form is likely preferable (the first form is actually loader.kboot's default).

There's also ways to pass in a mfs image, should that be desirable. We use

curl -o ${fn} <some url>
xz -d ${fn}
disk=${fn%%.xz}
ln -s ${disk} /tmp/wellknown.md
kboot hostdisk_override="$disk"

with a loader.conf in that has image that has

mdroot_load="YES"
mdroot_type="md_image"
mdroot_name="host:/tmp/wellknown.md"
vfs.root.mountfrom="ufs:/dev/md0"

in it.

loader.kboot has a number heuristics to guess the right thing to use, so it sees that the boot device should be this MD file. It's unclear how helpful these probing things are, and you may need to be more explicit with bootdev= if they get in the way (since any bootdev being specified overrides the probing).

[zdykstra]

A few questions:

• Is https://cgit.freebsd.org/src/tree/stand/kboot the source for loader.kboot ?
• Is this able to be compiled outside of the FreeBSD build process e.g. on a Linux host?
• Does the target boot pool need to exported prior to running loader.kboot ?
• Are there any limitations on pool feature flags / design that would prevent that pool from being used?

ZFSBootMenu is modular enough / we already have OS detection via /etc/os-release that I think I can probably stub in a super basic form of booting FreeBSD if I'm able to get a working loader.kboot built for Linux.

[bsdimp]

• Yes.
• Yes.[*] -- The whole tree is buildable, but I don't know if stand is w/o building lots of other things... will check
• I don't think so, but I'm unsure of the 'shutdown' that's done by Linux. loader.kboot opens the raw devices for the zfs pool. Alternatively, if the pool is mounted somewhere on the host, then the host: filesystem can be used to read the kernel, loader bits, etc (host: can have its own root since it also has special casing for /proc and /sys). In the latter case you'd need to set vfs.root.mountfrom.
• I don't think so... Except that if the pool is newer than the code that loader.kboot is built, it's possible it has features that aren't 'read-only safe' and thus it wouldn't be bootable.

I'll have to see how hard it is to build on Linux... I have a linux test machine that was just built for me that I need to log into for amd64 kboot work anyway... /stand is built such that the loader.kboot is a linux binary, no matter what... Some of the other loaders may be less tolerant of that, but 'can be built' might mean needing to build a few things piecemeal, then linking loader.kboot.... we do pull from bits of the FreeBSD kernel, OpenZFS sources, FreeBSD's libc, etc when building the boot loader stuff, so it would be a relatively full source tree...

[sydbarrett74]

Any updates on this issue? 😄

Bumping this up for attention. Is there any help needed to kickstart the work on this ?

[bsdimp]

I have LinuxBoot working for FreeBSD/arm64, but not for FreeBSD/amd64 yet.

I guess, what do you need from me to proceed?

[zdykstra]

I have LinuxBoot working for FreeBSD/arm64, but not for FreeBSD/amd64 yet.

I guess, what do you need from me to proceed?

Unfortunately, right now we realistically only support amd64. The most readily available arm64 hardware - RPI - has a number of problems that limits it's usefulness for us.

Once amd64 support is available, we'll be happy to start hacking on integrating that tool into ZBM!