datumctl build plugin: build unikernel OCI images and push to any registry

[scotwells]

Goal

Give users a familiar, container-style workflow for producing unikernel images: a new datumctl build plugin that builds a unikernel OCI image from an application's Dockerfile and pushes it to a registry of the user's choosing (Docker Hub, GHCR, a private registry, …), producing an image reference that deploys directly on Datum compute.

Why

Deploying on Datum compute today requires a unikernel image that was built off-platform with the Unikraft toolchain and pushed by hand. That means knowing tooling and packaging rules most users have never seen — plain docker build output does not boot on the runtime. Users coming from other providers expect build then push with their existing registry credentials to just work; the build plugin closes that gap without asking them to learn unikernel internals.

Requirements

• Familiar experience. Dockerfile in, image reference out: --tag/--push semantics that mirror docker build/docker push, with registry auth taken from the standard Docker credential config. Requiring a local Docker/BuildKit daemon is acceptable — that matches the experience of the tools users already run.
• Any registry. The user picks the push target; nothing assumes a Datum-operated registry. Private registries work end to end with the existing image pull credential flow at deploy time.
• Bootable output. The plugin owns the unikernel packaging rules (target platform, OCI index layout, PIE and image-size constraints) so images it produces boot on Datum compute, and constraint violations fail at build time with actionable errors instead of surfacing as boot failures.
• Standard plugin distribution. Ships through the datumctl plugin index (datumctl plugin install …) like the existing compute plugin.

Acceptance criteria

From the consumer's point of view:

• I can install the plugin with datumctl plugin install and get going from --help alone — no separate setup guide needed for a first build.
• Starting from my app and its Dockerfile, one command builds a unikernel image and pushes it to my registry — Docker Hub, GHCR, or my private registry — using the credentials from my existing docker login.
• The command prints the pushed image reference (tag and digest) ready to use in a deploy.
• Deploying a workload with that reference boots and serves traffic, with no manual packaging or fix-up steps between build and deploy.
• If my app can't run as a unikernel (unsupported binary, image too large, …), the build fails with a message that tells me what's wrong and what to change — I never find out via a boot failure.
• The same command works unattended in CI: non-interactive auth and exit codes that fail my pipeline when the build or push fails.
• A one-page quickstart takes me from an empty directory to a deployed hello-world.

Per-language guides and CI-tested working examples are tracked separately in #175; the quickstart above is part of this plugin's definition of done.

References

Authoring the plugin (datumctl):

• Building plugins — naming convention, context-injection env vars, the go.datum.net/datumctl/plugin Go SDK, checksum/trust requirements
• Using plugins — install/search/upgrade UX the plugin participates in
• Publishing catalogs and the official plugin index — manifest schema and submission flow
• Reference plugin example
• feat: datumctl compute plugin — deploy and manage workloads from the CLI #113 — the datumctl compute plugin, the in-repo precedent this plugin should feel consistent with

Unikraft image building:

• Unikraft CLI reference — the unikraft CLI (supersedes kraft cloud); build requires a Docker/BuildKit daemon
• Unikraft quick start — CLI install and first deploy
• Application catalog examples — Dockerfile-based reference apps
• Open-source Unikraft docs — unikernel concepts and the kraftkit toolchain

User-facing build guides to model ours on:

• Guides index, e.g. Node, Go, Python, Rust — per-language Dockerfile-to-running-app walkthroughs

[scotwells]

We validated this workflow end to end — it's buildable today

We took a plain Go HTTP service and walked the exact experience this issue describes: Dockerfile in, unikernel OCI image out, pushed to a private Docker Hub repo with ordinary registry credentials, then deployed through the existing image-pull-credential flow on the us-central-1-lab runtime. The microVM reached running in about 1.3 seconds and the pod went Running / Ready, serving traffic. No Datum-operated registry, no hosted build service — the entire build ran locally, which lines up with the "local Docker/BuildKit daemon is acceptable" requirement.

The single variable that decided success was the base runtime the app image is built against. Same app, same node:

Built against	What users see
Community (open-source) base runtime	App boots and even serves traffic, but the platform never marks it ready — the pod sits Pending forever
Unikraft enterprise base runtime (official/base)	running in ~1.3s, pod Running/Ready

That distinction is invisible in the image metadata and produces the worst kind of user experience (a deploy that half-works) — worth keeping in mind when deciding how much of the base-runtime choice the plugin exposes to users.

A lead worth evaluating: Unikraft's upstream Go libraries

The kraft CLI is a thin layer over upstream Go libraries (the kraftkit.sh module), and we confirmed from the source that Unikraft's own cloud deploy command runs the same local pipeline as plain packaging — there's no server-side build step to replicate. The pieces map closely onto this issue's requirements: their initrd package turns a Dockerfile into the guest filesystem via BuildKit (the docker build-familiar part), and their oci packager handles the base-runtime merge, platform stamping, and pushes using the user's standard Docker credential config (the "any registry" part). Whether the plugin embeds those libraries, shells out to the CLI, or takes another shape entirely is open for investigation — but staying on Unikraft-supported upstream code paths would mean inheriting their packaging rules rather than re-implementing them, and it gives us an upstream home for fixes (we already hit one runtime-name-resolution bug worth filing).

Constraints we learned the hard way (whatever the design ends up being)

• App binaries must be static PIE. The unikernel loader rejects anything else; a default go build produces a VM that dies ~1.6s after boot with a cryptic loader error. Good candidate for an up-front check with an actionable message.
• The runtime caches tag→digest per node and never re-resolves. Re-pushing a fixed image under the same tag silently redeploys the old bits; digest references (or unique tags) avoid it. Also relevant: base:latest upstream is a moving tag.
• The target platform must be kraftcloud/x86_64 in the OCI index — the toolchain stamps it natively when building against a base runtime that carries that platform.

The one product decision

Access to the enterprise base runtime is the whole ballgame: it's what carries the guest-side support the platform needs to mark instances running, and it's access-controlled on Unikraft's registry (we validated with vendor-issued credentials — self-service accounts can't pull it). That's a licensing/commercial conversation to have alongside the technical investigation; everything else in this issue's requirements list is proven with stock upstream tooling today.

A working reference of the full flow (Dockerfile with the static-PIE build, Kraftfile, exact packaging invocation) is up as a draft PR: datum-cloud/unikraft-provider#75.