-
Notifications
You must be signed in to change notification settings - Fork 108
Milestones
List view
Restructure the snapchain repo into a cargo workspace with `crates/` (packages) and `bin/` (apps), inspired by [reth](https://github.com/paradigmxyz/reth) and [Foundry](https://github.com/foundry-rs/foundry), so the repo can host the node, CLI, and future network monitoring tools side-by-side (similar in spirit to [farcasterxyz/hub-monorepo](https://github.com/farcasterxyz/hub-monorepo), but using cargo idioms). ## Target layout ``` snapchain/ ├── Cargo.toml # pure workspace + [workspace.dependencies] ├── bin/ │ ├── snapchain/ # node (was src/main.rs) │ ├── dump-wal/ # was src/bin/dump_wal.rs │ ├── follow-blocks/ # ...etc, one crate per current src/bin/* │ └── ... └── crates/ ├── proto/ # was top-level proto/ (package: snapchain-proto) └── snapchain/ # was top-level src/ (package: snapchain — kept to avoid renaming 61 use-sites) ``` ## Naming decisions (low-churn) - Lib crate package name stays **`snapchain`** so existing `use snapchain::...` imports do not change. - Node app crate is `snapchain-node` but its `[[bin]] name = "snapchain"`, preserving the executable name. - App-binary executable names keep underscores (`dump_wal`, `follow_blocks`, ...) so Dockerfile/scripts/ops references continue to work. ## Production safety invariants (MUST hold across every PR in this milestone) The auto-upgrade flow used by mainnet operators ([scripts/snapchain.sh](https://github.com/farcasterxyz/snapchain/blob/main/scripts/snapchain.sh), [scripts/snapchain-bootstrap.sh](https://github.com/farcasterxyz/snapchain/blob/main/scripts/snapchain-bootstrap.sh)) is **Docker-image-based, not binary-based**: it `curl`s text artifacts from `raw.githubusercontent.com/farcasterxyz/snapchain/@latest/...` and pulls `farcasterxyz/snapchain:latest` via `docker compose`. Nothing in the upgrade path fetches a binary tarball or runs `cargo install`. Production keeps working **iff** every PR in this milestone preserves all of the following: | Invariant | Why it matters | |---|---| | Published image stays **`farcasterxyz/snapchain:latest`** | `docker-compose.mainnet.yml` line 3 pulls this exact tag; renaming breaks every operator overnight. | | Image entrypoint stays **`./snapchain --id N`** | Dockerfile CMD line 74; achieved by `[[bin]] name = "snapchain"` on the node app crate. | | Compose service name stays **`snapchain`** | `scripts/snapchain.sh` does `$COMPOSE_CMD up -d snapchain`, `stop snapchain`, `logs ... snapchain`. | | Files at these exact root paths: **`scripts/snapchain.sh`**, **`docker-compose.mainnet.yml`**, **`validators.toml`**, **`grafana/grafana-dashboard.json`**, **`grafana/grafana.ini`** | The auto-upgrade script `curl`s each by hardcoded path. | | Binary executable names preserved: **`snapchain`**, **`setup_local_testnet`**, **`follow_blocks`**, **`submit_message`**, **`perftest`**, etc. | Dockerfile lines 42 and 65–71 reference these by name; preserved via `[[bin]] name = "<underscore_name>"` in each app crate. | | **CI path filters keep matching the lib + proto trees** | `docker-build.yml` and `verify.yml` use `paths: ['src/**', 'proto/**']`; after the restructure these silently stop matching, no new image gets published, prod nodes stop receiving updates. **Step 6 must update these.** | The only invariant that requires a deliberate code change to maintain is the last one (CI path filters) — see issue [#878](https://github.com/farcasterxyz/snapchain/issues/878) for details. ## Sequenced execution The work is tracked as the issues attached to this milestone, ordered for incremental, separately-mergeable PRs: 1. Adopt `[workspace.dependencies]` in place 2. Move `proto/` → `crates/proto/` 3. Extract `src/bin/*` binaries into `bin/*` crates 4. Extract node binary (`src/main.rs`) into `bin/snapchain/` 5. Move lib crate from `src/` → `crates/snapchain/` 6. Update Dockerfile, Makefile, and CI workflows for new layout 7. (Polish) Adopt `[workspace.package]` for shared metadata ## Process notes - **Concurrent merge friction.** Steps 5 (lib move) and 6 (CI/infra) conflict aggressively with any in-flight feature work in `src/`. Recommended: a soft freeze on merging changes that touch `src/` for the day step 5 lands, or have the step-5 author shepherd the rebase right before merge. Steps 1–4 are tolerant of concurrent merges. - **Post-migration cleanup (separate PR, not blocking).** After step 5 lands, run a `pub` visibility audit: items that were `pub` only because the (now-extracted) `main.rs` needed them are now legitimately part of the lib's public API. Some should be `pub(crate)` instead. `cargo doc --no-deps --open` shows the new public surface; this is a hygiene PR worth doing once the dust settles. - **Cargo-chef cache invalidation.** The Dockerfile uses `cargo chef`. Each restructure step changes the workspace manifest set, which invalidates the chef cache layer once. Expected — no action needed beyond letting CI rebuild the cache. ## Out of scope - Splitting the lib into multiple crates (`snapchain-storage`, `snapchain-network`, ...) — possible follow-up. - New tool/app crates (e.g. monitoring) — land after the restructure, in their own PRs. - Switching to a non-cargo monorepo tool — not needed at this scale. ## References - [reth](https://github.com/paradigmxyz/reth) — `bin/` + `crates/` cargo workspace, gold standard for Rust network nodes. - [Foundry](https://github.com/foundry-rs/foundry) — many CLI apps + shared `foundry-*` crates; closest analog to "node + CLI + monitoring tools". - [Lighthouse](https://github.com/sigp/lighthouse) — clean `[workspace.dependencies]` model.
No due date•0/7 issues closedThe next protocol release
No due date•3/6 issues closedThe next patch release
No due date•9/11 issues closed