v0.2.5: split BPF datapath into fast_path + finalize via bpf_tail_call

[lunarthegrey]

Summary

Fixes the v0.2.4 stack-budget regression on UniFi 5.15 kernels by splitting the BPF datapath into two programs connected by bpf_tail_call. Each program gets its own 512-byte stack budget. Same architecture establishes the pattern for future fast-path-internal stages (additional packet transforms, more sophisticated FIB logic) without re-bisecting stack bytes every release.

The proximate failure (reported on edge1-mci1-net): UniFi's 5.15.72-ui-cn9670 aarch64 kernel rejected v0.2.4's fast_path BPF program at combined stack size of 3 calls is 544. Too large — same bytecode that loaded cleanly on CI's qemu 5.15 vanilla x86_64 (stack depth 0+360+0+0). UniFi's BPF patches plus the aarch64 JIT account stack ~120 bytes higher than vanilla.

Architecture

                       packet ingress
                              │
                              ▼
   ┌──────────────────────────────────────────────────┐
   │ fast_path  (XDP, attached to eth0..ethN)         │  Frame A
   │   classification (allow-prefix, block-prefix)     │  fits 512B
   │   FIB lookup  (kernel-fib | custom-fib | compare)│
   │   devmap pre-check                                │
   │   TTL decrement / L2 rewrite (in-place)           │
   │   write per-CPU MUTATION_CTX                      │
   │   bpf_tail_call(MUTATION_PROGS, 0)  ──────────┐  │
   └────────────────────────────────────────────────│──┘
                                                    │
                                                    ▼
   ┌──────────────────────────────────────────────────┐
   │ finalize  (XDP, tail-called by fast_path)        │  Frame B
   │   read MUTATION_CTX                               │  fresh 512B
   │   mss-clamp lookup + (optional) MSS rewrite       │
   │   VLAN choreography (push / pop / rewrite)        │
   │   bpf_redirect_map(egress_ifindex)                │
   └──────────────────────────────────────────────────┘

mss-clamp + VLAN + redirect move from forward_success into finalize. Per-prefix LPM keys + TCP-options walk live in finalize's fresh stack budget. fast_path's responsibilities shrink to classification + L2/TTL, which fits comfortably under any kernel's accounting.

This is not the multi-module dispatcher (SPEC §3.4 / §5.0) — that's for chaining independent modules at the same hook (ddos in front of fast-path, sampler behind it). Tail-call is for splitting one logical pipeline. Both eventually exist; v0.2.5 ships only the former.

What's in the PR

Area	Files	Notes
New BPF program	crates/modules/fast-path/bpf/src/finalize.rs	#[xdp] pub fn finalize — reads MUTATION_CTX, runs mss-clamp + VLAN + redirect. ~280 LOC; mss-clamp + VLAN choreography moved here verbatim from main.rs.
BPF maps	maps.rs	New MutationCtx struct (16 bytes), MUTATION_CTX (PerCpuArray, single-element scratch), MUTATION_PROGS (ProgramArray, 8 slots). New StatIdx 35/36 (err_tail_call, err_mutation_ctx).
BPF program	main.rs	forward_success writes MutationCtx and tail-calls into MUTATION_PROGS[0] instead of doing mss-clamp + VLAN + redirect inline. ~440 LOC of mss-clamp + VLAN choreography moved out (now in finalize.rs).
Userspace lifecycle	linux_impl.rs	attach() loads finalize first → populates MUTATION_PROGS[0] with finalize's FD → loads + attaches fast_path. Order matters. New populate_mutation_progs helper. New tail_call_chain_from_pin for status reporting.
Pin lifecycle	pin.rs	New FINALIZE_PROGRAM_NAME constant + PROGRAM_NAMES array. MAP_NAMES grows to 19 (added MSS_CLAMP_V4/V6/BY_IFACE that were missing from v0.2.4, plus MUTATION_CTX/PROGS). pin_program_and_maps walks both program names.
CLI status	loader.rs	New "tail-call chain" section reports MUTATION_PROGS[0] occupancy. Stat names array updated for indices 33-36 (mss-clamp + tail-call diagnostics — bringing it in sync with what's actually defined).
Test harness	tests/common/mod.rs	Harness::new now loads both programs and populates MUTATION_PROGS[0] before returning. bpf_prog_test_run follows tail-calls (kernel re-enters its dispatcher for the target program), so existing tests transparently see the full chain's verdict + mutations.
Docs	README.md, new docs/runbooks/tail-call-architecture.md	Status table row for v0.2.5+; new runbook covers chain topology, MutationCtx wire format, debug commands (bpftool prog show, bpftool map dump MUTATION_PROGS), and how future stages slot in.
Version	Cargo.toml, VERSION, README install snippets	0.2.4 → 0.2.5

What's deliberately NOT in this PR

• Netns end-to-end integration test (tests/tail_call.rs was in the plan). The kernel BPF_PROG_TEST_RUN harness already exercises the tail-call via the existing fixtures (now updated to populate MUTATION_PROGS[0]). A real-veth + AF_PACKET capture test is good additional coverage but ~150 LOC of test infra and was the right thing to defer to keep this PR focused on the architecture change. Will land as a follow-up.
• bpf_fib_lookup per-CPU map move (mentioned as "alternative B" in earlier discussion). Tail-call obviates it; we have plenty of stack headroom now.
• Multi-stage chain (slots 1-7 of MUTATION_PROGS). Reserved capacity is in place for future use; no actual stages today.
• The dispatcher (SPEC §5.0). Different problem entirely — for ddos / sampler / randomizer composition.

CI expectations

Existing CI matrix should pass:

• ✅ fmt + clippy + test (workspace lib tests pass on macOS dev: 94 + 40)
• ✅ Cross-build matrix (4 targets) — userspace-only, no BPF concern
• ✅ qemu-verifier 5.15 + 6.6 — should load both programs, attach succeeds, sudo-gated test fixtures pass through the tail-call chain

The qemu jobs are the meaningful test for "does the verifier accept this on real kernels." If they pass, vanilla 5.15 / 6.6 are fine. UniFi-style stricter accounting will be confirmed via post-merge deployment on the same router that hit the v0.2.4 regression.

Test plan

Pre-merge (CI):

• cargo fmt --all --check clean
• cargo clippy --workspace --all-targets --all-features -- -D warnings clean
• cargo test --workspace --lib 94 + 40 pass on macOS dev host
• CI fmt+clippy+test passes
• CI cross-build (4 targets) passes
• CI qemu-verifier 5.15 + 6.6 passes (with updated harness following the chain)

Post-merge on the deployed UniFi router:

• apt install ./packetframe_0.2.5_arm64.deb. sudo systemctl restart packetframe.
• sudo packetframe feasibility --config /etc/packetframe/packetframe.conf --human — all xdp.attach.ethN now PASS (no more 544/512 rejection).
• sudo packetframe status — the new "tail-call chain" section reports MUTATION_PROGS[0]: populated (finalize).
• Add a per-prefix mss-clamp directive: mss-clamp 23.191.200.0/24 1360. sudo packetframe reconfigure.
• tcpdump -i eth2 -n 'tcp[tcpflags] & tcp-syn != 0' -vv confirms wire MSS=1360 on outbound SYNs.
• sudo packetframe status | grep mss_clamp_applied shows the counter climbing.
• err_tail_call and err_mutation_ctx stay at 0.

Tag flow after merge

git checkout main && git pull
git tag -a v0.2.5 -m "v0.2.5"
git push origin v0.2.5

(The version bump is in this PR — same pattern as v0.2.4 — so just tag and push.)

🤖 Generated with Claude Code