|
| 1 | +# Copilot Instructions — rbft (RBFT QBFT Node) |
| 2 | + |
| 3 | +## Architecture Overview |
| 4 | + |
| 5 | +This is a **QBFT Byzantine Fault Tolerant consensus** implementation for the RBFT EVM network, |
| 6 | +built as a custom consensus plugin on top of [reth](https://github.com/paradigmxyz/reth) v1.10.1. |
| 7 | + |
| 8 | +### Crate Responsibilities |
| 9 | + |
| 10 | +| Crate | Role | |
| 11 | +|---|---| |
| 12 | +| `crates/rbft` | Pure QBFT state machine — a near-1:1 translation of `doc/qbft-spec/dafny/` | |
| 13 | +| `crates/rbft-node` | Reth node binary; plugs `rbft` into reth via the Engine API and RLPx | |
| 14 | +| `crates/rbft-utils` | CLI tooling: genesis generation, testnet orchestration, `logjam` log analyser | |
| 15 | +| `crates/rbft-megatx` | Transaction spam tool for load testing | |
| 16 | +| `crates/rbft-validator-inspector` | TUI for monitoring a running testnet | |
| 17 | + |
| 18 | +### Data Flow |
| 19 | + |
| 20 | +``` |
| 21 | +rbft::NodeState ──(QbftMessages)──► rbft_consensus.rs ──(Engine API)──► reth |
| 22 | + ▲ │ |
| 23 | + │ ▼ |
| 24 | + └─── node_auxilliary_functions.rs rbft_protocol.rs ◄──► RLPx peers |
| 25 | +``` |
| 26 | + |
| 27 | +- `rbft::node` (`node.rs`) implements `upon_*` transition functions from the Dafny spec. |
| 28 | +- `rbft_consensus.rs` is the integration layer: it drives the state machine, calls |
| 29 | + `new_payload` / `fork_choice_updated` on reth's Engine API, and relays QBFT messages over RLPx. |
| 30 | +- `rbft_consensus/on_chain_config.rs` reads the live validator set from the on-chain |
| 31 | + `QBFTValidatorSet` contract (storage slot layout matters — don't change it without updating |
| 32 | + `genesis.rs`). |
| 33 | +- Validator selection at each epoch: the block hash is used as a random seed to deterministically |
| 34 | + select a subset when `max_validators < total_validators`. |
| 35 | + |
| 36 | +## Critical Developer Workflows |
| 37 | + |
| 38 | +### Never compile directly — always use Makefile targets |
| 39 | + |
| 40 | +The node requires a genesis file and pre-built assets before it can run. |
| 41 | + |
| 42 | +```bash |
| 43 | +make testnet_load_test # Full end-to-end build + run + TPS check (CI gate) |
| 44 | +make testnet_start # Start 4-node local testnet (persistent) |
| 45 | +make test # cargo test --all |
| 46 | +make fmt # cargo fmt --all |
| 47 | +make clippy # cargo clippy -D warnings |
| 48 | +``` |
| 49 | + |
| 50 | +`rustfmt.toml` enables unstable options, so a nightly toolchain must be the |
| 51 | +active default (or set via `rustup override`) for `cargo fmt` to succeed. |
| 52 | + |
| 53 | +### Pre-commit checklist (also required before pushing) |
| 54 | + |
| 55 | +```bash |
| 56 | +cargo fmt --all |
| 57 | +python scripts/check_line_length.py # max 100 chars; fails CI if violated |
| 58 | +cargo test |
| 59 | +cargo clippy --all-targets --all-features -- -D warnings |
| 60 | +make testnet_load_test |
| 61 | +``` |
| 62 | + |
| 63 | +### Testnet assets location |
| 64 | + |
| 65 | +- Assets (keys, genesis, enodes): `~/.rbft/testnet/assets/` |
| 66 | +- Logs: `~/.rbft/testnet/logs/node*.log` |
| 67 | +- DB: `~/.rbft/testnet/db/` |
| 68 | + |
| 69 | +Use `make logjam` (or `RBFT_LOGJAM_QUIET=1 RBFT_LOGJAM_FOLLOW=1 make logjam`) to analyse logs in |
| 70 | +chronological order with a message-delivery histogram. |
| 71 | + |
| 72 | +### Key environment variables |
| 73 | + |
| 74 | +| Variable | Default | Purpose | |
| 75 | +|---|---|---| |
| 76 | +| `RBFT_NUM_NODES` | 4 | Nodes in testnet/genesis | |
| 77 | +| `RBFT_BLOCK_INTERVAL` | 0.5 | Block interval (seconds, float) | |
| 78 | +| `RBFT_BASE_FEE` | 4761904761905 | Genesis base fee (wei) | |
| 79 | +| `RBFT_EXIT_AFTER_BLOCK` | — | Auto-exit testnet at this height | |
| 80 | +| `RBFT_RESEND_AFTER` | unset (disabled) | Resend cached messages after N seconds stall; **`0` is not disabled** — it causes continuous resends | |
| 81 | +| `RBFT_FULL_LOGS` | false | Log on every advance cycle | |
| 82 | + |
| 83 | +## Project-Specific Conventions |
| 84 | + |
| 85 | +- **`node_auxilliary_functions.rs` must mirror the Dafny spec**: variable names use `snake_case` |
| 86 | + but otherwise stay as close as possible to `doc/qbft-spec/dafny/`. Do not refactor for style. |
| 87 | +- **Sets are `Vec`/slices, not `HashSet`**: deliberately matches the spec; don't change without |
| 88 | + careful thought about ordering semantics. |
| 89 | +- **Custom timestamp validation**: `consensus_builder.rs` overrides reth's timestamp check to |
| 90 | + allow equal timestamps between parent/child (needed for sub-second blocks). |
| 91 | +- **RLPx sub-protocol**: QBFT messages are sent over a custom RLPx capability defined in |
| 92 | + `rbft_protocol.rs`, not over the standard Eth wire protocol. |
| 93 | +- **On-chain config is authoritative**: validator set, block interval, and epoch length are read |
| 94 | + from the `QBFTValidatorSet` contract at each epoch boundary — not from config files. |
| 95 | +- **`RbftConfig`** (in `rbft-utils/src/types.rs`) is embedded in `genesis.json` under the |
| 96 | + `rbft` key and governs message-buffer sizes, reconnect backoff, and relay behaviour. |
| 97 | + |
| 98 | +## `rbft_consensus` Sub-modules |
| 99 | + |
| 100 | +`crates/rbft-node/src/rbft_consensus/` contains the reth integration layer, split across: |
| 101 | + |
| 102 | +| File | Purpose | |
| 103 | +|---|---| |
| 104 | +| `rbft_consensus.rs` | Main consensus loop: drives `NodeState`, calls `new_payload` / `fork_choice_updated`, relays messages | |
| 105 | +| `rbft_protocol.rs` | Custom RLPx capability (`RbftProtocolHandler`); manages per-peer `Connection` state, message cache, idle-timeout disconnects | |
| 106 | +| `on_chain_config.rs` | Reads `QBFTValidatorSet` contract storage at epoch boundaries; resolves enode hostnames via DNS | |
| 107 | +| `consensus_builder.rs` | `RbftBeaconConsensus` wrapping `EthBeaconConsensus`; overrides `validate_header_against_parent` to allow `child.timestamp == parent.timestamp` | |
| 108 | +| `payload_validator.rs` | `RbftPayloadValidator` wrapping `EthereumExecutionPayloadValidator`; delegates `ensure_well_formed_payload` then calls `try_recover` | |
| 109 | +| `validator_builder.rs` | `RbftPayloadValidatorBuilder` — wires `RbftPayloadValidator` into reth's RPC add-ons via `PayloadValidatorBuilder` | |
| 110 | +| `aligned_interval.rs` | Timer that fires on wall-clock-aligned boundaries (keeps block cadence in sync across nodes) | |
| 111 | + |
| 112 | +`RbftNode` (in `rbft_node.rs`) assembles these into reth's `ComponentsBuilder` using |
| 113 | +`RbftConsensusBuilder` and `RbftPayloadValidatorBuilder`. |
| 114 | + |
| 115 | +## Metrics & Health |
| 116 | + |
| 117 | +`crates/rbft-node/src/metrics.rs` exposes a `warp` HTTP server whose port is derived |
| 118 | +from the main HTTP port: `metrics_port = 9000 + (http_port - 8545)` (e.g. `8545` → port `9000`): |
| 119 | + |
| 120 | +- `GET /metrics` — Prometheus text format (`qbft_block_height`, `qbft_round`, `qbft_is_proposer`) |
| 121 | +- `GET /health` — JSON `HealthStatus`; returns `503` when no block has been committed within |
| 122 | + `max(expected_block_interval_ms × 5, 30s)` where `expected_block_interval_ms` defaults to |
| 123 | + `1000ms` (it is not currently updated from the on-chain `block_interval`) |
| 124 | + |
| 125 | +`QbftMetrics` is `Arc`-shared across the consensus loop and the HTTP server. Update it via |
| 126 | +`record_consensus_state(height, round, is_proposer)` on every committed block. |
| 127 | + |
| 128 | +## Key Files |
| 129 | + |
| 130 | +- `crates/rbft/src/node.rs` — QBFT `upon_*` state transition functions |
| 131 | +- `crates/rbft/src/node_auxilliary_functions.rs` — cryptographic primitives, quorum logic |
| 132 | +- `crates/rbft-node/src/rbft_consensus.rs` — main consensus loop (2500+ lines) |
| 133 | +- `crates/rbft-node/src/rbft_consensus/rbft_protocol.rs` — RLPx wire protocol |
| 134 | +- `crates/rbft-node/src/rbft_consensus/on_chain_config.rs` — live validator set reader |
| 135 | +- `crates/rbft-node/src/metrics.rs` — Prometheus + health endpoint |
| 136 | +- `crates/rbft-utils/src/genesis.rs` — genesis + contract deployment (uses `revm` inline) |
| 137 | +- `contracts/QBFTValidatorSet.sol` — on-chain validator registry |
0 commit comments