Skip to content

IMPLEMENTATION_PLAN.md

Latest commit

 

History

History
117 lines (92 loc) · 3.63 KB

IMPLEMENTATION_PLAN.md

File metadata and controls

117 lines (92 loc) · 3.63 KB

Implementation Plan (Tokio + Single-Owner Core)

This is the Step 1 plan based on the current design docs. The aim is an idiomatic Rust structure with a single-owner core loop and explicit message passing.

1) Crate Layout and Responsibilities

  • config
    • Parse config + LAN/WAN profiles + limits.
    • Load peer list, storage paths, ops endpoints.
  • protocol
    • FrameHeader encode/decode.
    • Message type constants + flags.
    • Payload codecs (protobuf/packed) kept separate.
  • net
    • TCP accept/connect and per-connection tasks.
    • Reader: parse header, read payload, emit CoreEvent::InboundFrame.
    • Writer: drain priority queues (0..3) with backpressure.
  • core
    • Single-owner event loop for consensus, WAL, state machine.
    • Owns RaftState, Wal, KvState, LeaseTable, WatchRegistry.
  • raft
    • Role transitions + leader/follower logic + backtrack hints.
  • log
    • WAL segment manager, record encoding/decoding, recovery scan.
    • Fsync policies (always / group_commit / never).
  • snapshot
    • Snapshot read/write format and install protocol.
  • state
    • Deterministic apply rules for PUT/DEL/TXN/LEASE_*.
  • watch
    • Watch registry, bounded queues, BEHIND policy.
  • client
    • Client request handling + RESP encoding + dedupe cache.
  • ops (optional)
    • HTTP endpoints: health/ready/metrics/status.

2) Core Ownership Model

NodeCore is the single mutable owner of all consensus-critical state:

  • Consensus/role state: role, term, voted_for, next_index, match_index
  • WAL handle and metadata
  • KV state + global revision + key revisions
  • Lease table and expiry queue
  • Watch registry + per-watcher queues
  • Commit/apply indices

Everything else sends events to the core via bounded channels.

3) Task Model (Tokio)

  • Listener task
    • Accepts TCP connections, spawns per-connection tasks.
  • Connection reader
    • Parses FrameHeader, reads payload, sends to core.
  • Connection writer
    • Consumes outbound frames with priorities.
  • Core task
    • Single event loop receives CoreEvents from net/timers/client.

4) Events and Channels

Use bounded tokio::mpsc for explicit backpressure.

CoreEvent (examples)

  • InboundFrame { peer_id, header, payload }
  • ClientRequest { client_id, request_id, msg }
  • TickHeartbeat
  • TickElection
  • WalAppended { index }
  • SnapshotComplete { last_included_index }

Outbound

  • NetSend { peer_id, frame, priority }
  • ClientResponse { client_id, resp }

5) Event Loop (High-Level)

  • Inbound APPEND
    • Validate term, reset election timer, check prev_log.
    • Append to WAL, update commit_index.
    • Apply committed entries, ACK with match_index.
  • Client PUT/DEL/TXN
    • Append WAL entry, replicate, wait for quorum.
    • Fsync per policy, apply, RESP.
  • Heartbeats
    • Periodic APPEND(no entries) to followers.
  • Elections
    • On timeout, transition to candidate, send VOTE_REQ.

6) Offloading Heavy Work

Keep correctness in core, offload heavy work to background tasks:

  • Snapshot encoding/writing.
  • Optional compression / checksum verification.

Background tasks return results to core via events.

7) Initial Implementation Order (low-to-high risk)

  1. protocol (FrameHeader + constants)
  2. net (connection read/write, priority queues)
  3. core skeleton (event loop + channel wiring)
  4. log (WAL append + recovery scan)
  5. raft (roles + election timers)
  6. state (apply rules)
  7. client (RESP + dedupe)
  8. snapshot (install + compaction)
  9. watch (queue + BEHIND)
  10. ops (metrics, health, status)

This plan is intended to be idiomatic, minimal in shared mutability, and easy to reason about during interviews.