Skip to content

Embedding: kubo-free boot (in-process CID + local content store) #583

Description

@lthibault

Summary

Make ww boot fully independent of a local/network kubo daemon: compute the merged image root CID in-process and serve guest file blocks from a local content store, so no ipfs add / MFS dag_merge / IPNS resolve / CidTree-over-kubo happens on the boot path.

This is the deferred portion of infra-crunch slice 2. The crunch itself (issue-3 decision 3A) only makes the failure mode safe: missing/slow kubo now degrades (retry + backoff + /readyz=false, never process exit) and startup ordering guarantees kubo is up first. Boot still uses kubo on the happy path. Removing that dependency is this issue.

Why it's the embedding program, not a crunch slice

Boot currently touches kubo in four places (verified 2026-07-21):

  • crates/cell/src/image.rs:222add_dir each local layer to kubo to get its CID.
  • crates/cell/src/image.rs:210resolve_ipns_to_ipfs for /ipns/ layers.
  • crates/cell/src/image.rs:60dag_merge merges layers into the root CID via kubo MFS (MfsNamespaceGuard, client.mfs()).
  • crates/cell/vfs::CidTree (consumed src/executor.rs:71) — resolves guest file content by CID through kubo.

There is no in-repo UnixFS/dag-pb builder today (only raw blake3 crates/schema-id::compute_cid). So kubo-free boot requires (a) a local UnixFS/dag-pb CID computation and (b) a local blockstore the CidTree VFS reads from — which is precisely the in-process content store the embedding program is scoped to build. Doing it inside the reliability crunch would violate the "embedding never blocks crunch" discipline and drag the fragmented Rust BitSwap ecosystem risk (embedding spike c) onto the critical path.

Scope of this issue (the embedding program's first concrete win)

  • In-process merged-root-CID computation (local UnixFS/dag-pb), replacing the kubo add_dir + MFS dag_merge for baked local layers.
  • Local content store backing CidTree so guest file reads don't require kubo.
  • Boot mounts baked image layers from disk/local store with zero network IPFS and zero kubo dependency.
  • Ties into the larger embedding direction: one libp2p swarm (wetware's), block exchange as a ww behaviour, kubo exits the node.

Gate / sequencing

Starts after the infra crunch ships (per the accelerated embedding gate: p2p-layer dedup alone justifies starting post-crunch). Never blocks the crunch. Gets its own eng review.

References

  • Design doc: ~/.gstack/projects/wetware-ww/lthibault-review-synapse-design-tradeoffs-design-20260721-110737.md (Approach B, embedding program).
  • gbrain: projects/wetware-ww/infra-crunch-embedded-ipfs-direction-20260720.
  • Spike (a) local results confirmed the boot hard-fails without kubo (Failed to add local layer to IPFS … Connection refused).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions