Optionally build skywire as a gokrazy appliance (VPN mesh-router image)

[0pcom]

Track making skywire optionally buildable as a gokrazy appliance — a read-only, userland-less, single-purpose Go image — targeting the ethernet VPN mesh-router use case first (most skywire boards are wired). The normal .deb/AUR/MSI builds stay untouched; the appliance is a separate build/packaging path.

Why it fits

• skywire is a single static Go binary; apps are subcommands — exactly gokrazy's model. The visor is a well-behaved supervised daemon.
• The data plane is already Go-native: tun via songgao/water, and the vpn-router's DHCP+DNS is literally the vendored gokrazy router7 (pkg/vpnrouter/router7).
• skywire autoconfig is an in-process Go subcommand, so first-boot config can run without a shell.

The remaining work is the control plane (shell-outs to ip/iptables/hostapd) plus image/kernel/board bring-up.

Phases

Phase 0 — Go-native control plane (prereq for everything)

• 0a — ip → netlink + sysctl → /proc — new pkg/vpn/netctl. Behavior-identical to ip, removes iproute2 dependency + per-command pkexec. Validated against real netlink in a rootless netns. → feat(vpn): Go-native control plane — netlink (default) + nftables firewall (tag-gated) #3556
• 0b — iptables → google/nftables (masquerade, FORWARD accept, mesh-gateway REDIRECT, TCPMSS clamp). The delicate part: the --clamp-mss-to-pmtu expression (tcp option maxseg size set rt mtu) and REDIRECT need careful translation + validation. Alternatively, an interim path bundles the nft binary in the image.

Phase 1 — Pi4 ethernet-router appliance image

• A gok config (separate repo, mirroring apt-repo/AUR) referencing cmd/skywire as a gokrazy package.
• Kernel with tun + nf_tables/nft_nat/nft_masq enabled.
• First-boot config seeded to gokrazy /perm (keys + skywire.json) via in-process autoconfig.
• Scope: ethernet-uplink + ethernet-downstream VPN router + mesh gateway (no WiFi → no hostapd). This is the first genuinely shippable skywire appliance.

Phase 2 — WiFi (deferred; the hard part)

• No pure-Go hostapd. Bundle hostapd + wpa_supplicant (+ libnl/openssl) as image extra-files, plus the mac80211/cfg80211 stack + radio driver + firmware in the kernel. WiFi-uplink and WiFi-AP-downstream variants.

Phase 3 — Board bring-up + OTA (deferred)

• Pi4 is first-class gokrazy (arm64); Orange Pi Prime (Allwinner) is not — needs u-boot + mainline kernel + gokrazy boot integration (requires flashing a test board).
• Updates: gokrazy's own OTA (gok update); optionally serve gokrazy images over dmsg (fits the dmsg-served-apt-repo pattern).

Remaining control-plane shell-outs (the concrete surface)

• ip ×27 → netlink ✅ (0a)
• sysctl ×2 → /proc ✅ (0a)
• iptables ×9 → nftables ⬜ (0b)
• nmcli ×2 (vpn-client DNS) — appliance uses the mesh-gateway resolver instead; only invoked when DNSAddr is set. Minor.
• hostapd ×2 → bundle (Phase 2)

"Optional" structure

Make netlink/nftables the default on Linux (strictly better — also fixes the pkexec flood); gate only no-elevation exec behind a build tag if needed. Keep the gok image definition in its own repo. Normal builds unaffected.

Design + surface analysis produced while implementing 0a; 0b + Phase 1 are the immediate next steps, 2 + 3 are the deferred phases needing hardware.

[0pcom]

Progress (PR #3556):

• ✅ 0a — ip→netlink + sysctl→/proc (pkg/vpn/netctl), wired + default, netns-validated.
• ✅ 0b backend — nftables firewall (pkg/vpn/netctl/firewall_nft_linux.go, tag nftfw): masquerade / FORWARD / mesh REDIRECT / TCPMSS-clamp (native rt tcpmss), netns-validated incl. end-to-end REDIRECT. Not yet wired into the router/client firewall calls — that dispatch (behind nftfw, default stays iptables) lands with Phase 1 so the whole appliance build is tested together.

Next: Phase 1 — the gok eth-router image (separate repo), Pi4 kernel with tun+nftables, config seeded to /perm, and the firewall-call wiring behind nftfw.

[0pcom]

Phase 1 — build milestone validated ✅ (no hardware needed)

gok (gokrazy's tool) cross-compiled github.com/skycoin/skywire/cmd/skywire for arm64 (CGO_ENABLED=0), fetched the Pi kernel (gokrazy/kernel.rpi) + firmware + gokrazy essentials (dhcp/ntp/randomd/heartbeat/breakglass/serial-busybox), and assembled a 227 MB bootable image archive. So skywire builds as a gokrazy package and packs into a Pi4 image — the appliance approach works end to end.

Working starter config (gok -i skywire-router new, then):

{
    "Hostname": "skywire-router",
    "Environment": ["GOOS=linux", "GOARCH=arm64", "GOFLAGS=-tags=nftfw"],
    "Packages": [
        "github.com/skycoin/skywire/cmd/skywire",
        "github.com/gokrazy/serial-busybox",
        "github.com/gokrazy/breakglass"
    ],
    "PackageConfig": {
        "github.com/skycoin/skywire/cmd/skywire": { "CommandLineFlags": ["visor"] }
    }
}

Build: gok -i skywire-router overwrite --gaf skywire-router.gaf (or --full <device> to flash an SD card).

Remaining Phase 1 (needs #3556 merged + a board):

• Wire the router/client firewall calls through netctl.NFTFirewall behind nftfw (so the arm64 image's firewall is the Go-native nftables backend, not the absent iptables). The image above builds the released skywire; once feat(vpn): Go-native control plane — netlink (default) + nftables firewall (tag-gated) #3556 lands, the config picks up the netlink/nftables control plane.
• Seed first-boot config (keys + skywire.json) into gokrazy /perm via in-process autoconfig; kernel already has tun + nf_tables (validated it loads the rpi kernel's loadable modules).
• Flash a Pi4 and confirm it routes (the on-hardware step).

The gok config likely wants its own repo (like apt-repo/AUR); captured here for now.

[0pcom]

Validation update:

• netctl (0a) is now validated against the real kernel (rootless netns) across both a dummy and a real POINTOPOINT tun (the vpn apps' actual interface type) — every op: addr add/flush/del, link up + MTU, route add/replace/del via gateway, default-route-out-dev + iif policy rule, forwarding. Since netlink is exactly what ip issues to the kernel, this covers the control-plane behavior.
• nftables firewall (0b backend) validated against the real kernel incl. end-to-end REDIRECT.

On-board integration test — attempted, currently blocked from this host: the LAN SBCs aren't SSH-reachable (mesh/dmsgpty only), their PKs can't be mapped to LAN hosts from the transport list (751 transports, no addresses exposed), and the public vpn-server list is timing out on SD fetch. So the full vpn-router-on-a-Pi traffic test still needs a board I can reach + a known server. The netns real-kernel validation stands in for control-plane correctness in the meantime.

Next code step: wire the router/client firewall calls through netctl.NFTFirewall behind nftfw (a build-tag-gated firewall seam — default stays iptables, so the fleet is unaffected; only the appliance build uses nft). This completes 0b and is what makes the arm64 gok image's firewall functional.

[0pcom]

On-hardware validation done ✅ — ran the netctl + nftables tests on a real Orange Pi Prime (aarch64, 5.15.48-sunxi64) over the mesh (dmsg scp + pty exec), in an isolated netns: TestNFTFirewall, TestNetctlRealTUN, TestNetctlNetns all PASS. Both the netlink control plane (0a) and the nftables firewall (0b) work on the real SBC kernel. Board's own networking untouched (netns-isolated). Fleet-safety gap for #3556 closed.

[0pcom]

0b wiring done ✅ (#3558): the router's firewall rules now dispatch through a firewall backend seam — iptables by default (byte-identical to before, fleet unchanged), netctl.NFTFirewall under nftfw (the appliance's Go-native firewall). Both tag paths build + lint clean; vpn tests pass. So the arm64 gok image's firewall is now functional. Remaining Phase-0/1: seed /perm config on first boot + flash a Pi4 for the full router traffic test. (Deferred: mesh-client OUTPUT rules + server masquerade — not on the eth-router path.)