Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions experimental/starrocks/docs/TUNABLES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# CN tunables

Environment variables in the Sirius StarRocks CN's validated transport
registry. Most operators should not need to set these by hand.

## How they work

Transport knobs live in one registry, [`src/tunable.rs`](../src/tunable.rs).
They are resolved once at bring-up: a typo or out-of-range value **fails CN
startup** (it is never clamped or silently ignored). The CN then logs the
resolved set. That line is what the process actually got, not what the
launcher echoed.

Unset means the compiled default. Empty string is treated as unset.

The transport that reads these knobs lands in later PRs; until then the
registry only validates and logs.

## Transport (validated registry)

| Knob | Role |
|---|---|
| `SIRIUS_CN_RPC_TIMEOUT_SECS` | How long a CN waits for a peer RPC (lease, metadata). Raise this before treating a large-SF timeout as a query bug, since a busy peer can sit behind this bound. |
| `SIRIUS_CN_NIXL_XFER_TIMEOUT_SECS` | How long one nixl WRITE may take. Distinguishes a stuck fabric from a busy peer. |
| `SIRIUS_CN_NIXL_CANARY_BYTES` / `_FLOOR_GBPS` | First-contact bandwidth probe. A slow link is refused so a silent staged-copy fallback cannot look like a healthy transfer. `0` on the floor disables the check. |
| `SIRIUS_CN_NIXL_WARMUP_TIMEOUT_SECS` / `_EXPECT_PEERS` | Bring-up session warmup. The timeout is a budget, not a hard fail; expect-peers ends the loop early once that many peers are up. |
2 changes: 2 additions & 0 deletions experimental/starrocks/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,13 @@ mod proto;
mod prpc;
mod result_encoder;
mod result_store;
mod tunable;

pub use brpc::BrpcServer;
#[cfg(feature = "sirius-engine")]
pub use engine::SiriusEngine;
pub use fragment_executor::{FragmentExecutor, FragmentResult, StubExecutor};
pub use tunable::Tunables;

const COMPUTE_NODE_PROC_PATH: &str = "/compute_nodes";

Expand Down
8 changes: 7 additions & 1 deletion experimental/starrocks/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use sirius_starrocks_cn::SiriusEngine;
use sirius_starrocks_cn::StubExecutor;
use sirius_starrocks_cn::{
BackendServer, BrpcServer, ComputeNodeConfig, FeConfig, FragmentExecutor, HeartbeatServer,
SharedHeartbeatState, register_node, report_to_frontend_once, start_backend_server,
SharedHeartbeatState, Tunables, register_node, report_to_frontend_once, start_backend_server,
start_heartbeat_server,
};
use tokio::task::{JoinError, JoinSet};
Expand Down Expand Up @@ -64,6 +64,12 @@ impl Args {
/// Starts the CN listeners, registers with FE, and waits for shutdown.
#[instrument(name = "compute_node", skip_all)]
async fn run(self) -> Result<()> {
// FIRST, before a port is bound or a GPU pool is reserved: read and validate every
// transport tunable, and log what this CN actually got. A rejected value fails startup
// here rather than surfacing as an unexplained timeout mid-sweep, and the log line is
// the ground truth for the knobs.
Tunables::resolve().map_err(|err| anyhow!("invalid CN transport tunable: {err}"))?;

// Build the fragment executor before serving any RPC. Compiled with the engine, this brings
// up the GPU engine on its dedicated thread (fail-fast: a bad config or GPU failure exits
// before FE can route work here); otherwise it is a stub. The handle is held for the
Expand Down
Loading
Loading