Skip to content

perf: speed up rust fast-test server#5059

Open
lucarlig wants to merge 9 commits into
mainfrom
user/luca/fast-test-server-rust-benchmark
Open

perf: speed up rust fast-test server#5059
lucarlig wants to merge 9 commits into
mainfrom
user/luca/fast-test-server-rust-benchmark

Conversation

@lucarlig
Copy link
Copy Markdown
Collaborator

@lucarlig lucarlig commented Jun 4, 2026

Summary

This draft updates the Rust fast-test-server to reduce MCP benchmark latency:

  • replace per-call async mutex stats with relaxed atomic counters
  • route /mcp through a lean direct Streamable HTTP JSON-RPC handler for this fixed tool surface
  • preserve schema fixture tool metadata, including outputSchema
  • preserve MCP echo delay and jitter behavior on the direct handler
  • validate required direct-handler tool arguments and return JSON-RPC invalid params for malformed calls
  • validate direct-handler MCP sessions, including rejecting missing sessions with HTTP 400, unknown/deleted sessions with HTTP 404, and removing sessions on DELETE /mcp
  • advertise the latest RMCP protocol version (2025-11-25) while accepting older initialize payloads such as 2024-11-05
  • align the Go fast-time /version endpoint with mcp-go's latest protocol version and pin that in its unit test
  • switch Rust fast-test timezone handling to chrono-tz so IANA zones, DST, half-hour offsets, and UTC Z formatting match Go fast-time behavior
  • add Rust handler-level tests for version reporting, backward-compatible initialize, Streamable HTTP session lifecycle behavior, and Go-compatible timezone conversion
  • add a live ContextForge E2E parity test that discovers the federated Go/Rust tools, compares deterministic convert_time outputs, and checks Rust latency stays in the same CF-routed band as Go
  • update fast-test-server Rust dependencies to the latest crates.io versions not newer than 7 days as of 2026-06-04
  • remove unused RMCP streamable HTTP features and tokio-util from this server

Benchmark with sdk-benchmark from sco3/mcp-benchmark, single user, -i 100 -r 10000, get_system_time with {"timezone":"UTC"}:

Rust after changes:
Init:      1377.54 req/s, 0.73ms
Tool/list: 7930.87 req/s, 0.12ms
Tool call: 11050.63 req/s, 0.09ms

For context, the local Go fast-time run on the same benchmark shape measured:

Init:      1019.79 req/s, 0.98ms
Tool/list: 5209.63 req/s, 0.18ms
Tool call: 7403.40 req/s, 0.13ms

Verification

  • go test ./... in mcp-servers/go/fast-time-server
  • cargo fmt --check in mcp-servers/rust/fast-test-server
  • cargo test in mcp-servers/rust/fast-test-server
  • cargo build --release in mcp-servers/rust/fast-test-server
  • JWT_SECRET_KEY=my-test-key-but-now-longer-than-32-bytes MCP_CLI_BASE_URL=http://127.0.0.1:8080 .venv/bin/python -m pytest tests/live_gateway/mcp/test_fast_server_go_rust_parity.py -q -s
    • local ContextForge gateway with locally registered Go fast-time and Rust fast-test upstreams
    • deterministic convert_time parity passed for America/New_York DST and Asia/Kolkata half-hour offset cases
    • CF-routed latency parity passed with Go avg 0.025996s, Rust avg 0.026905s, FAST_SERVER_PARITY_CALLS=5
  • Live MCP smoke test against 127.0.0.1:19080:
    • initialized with 2024-11-05 request payload and received 2025-11-25
    • valid tools/list with issued session returned HTTP 200
    • missing session returned HTTP 400
    • fabricated session returned HTTP 404
    • deleted session returned HTTP 404

Note: make testing-up could not be used in this local environment because required GHCR images returned 403, so the live parity verification above used local CF and local upstream processes registered through the same /gateways API path.

Closes #4497

@lucarlig lucarlig self-assigned this Jun 4, 2026
@lucarlig lucarlig force-pushed the user/luca/fast-test-server-rust-benchmark branch 3 times, most recently from 8c30fbb to 7cac5dd Compare June 4, 2026 13:55
Signed-off-by: lucarlig <luca.carlig@ibm.com>
@lucarlig lucarlig force-pushed the user/luca/fast-test-server-rust-benchmark branch from 7cac5dd to 8a26277 Compare June 5, 2026 08:12
lucarlig added 4 commits June 5, 2026 10:44
Signed-off-by: lucarlig <lucarlig@protonmail.com>
Signed-off-by: lucarlig <lucarlig@protonmail.com>
Signed-off-by: lucarlig <lucarlig@protonmail.com>
Signed-off-by: lucarlig <lucarlig@protonmail.com>
@lucarlig lucarlig marked this pull request as ready for review June 5, 2026 09:18
@lucarlig lucarlig requested a review from msureshkumar88 June 5, 2026 09:18
lucarlig added 2 commits June 5, 2026 11:48
Signed-off-by: lucarlig <lucarlig@protonmail.com>
Signed-off-by: lucarlig <lucarlig@protonmail.com>
Copy link
Copy Markdown
Collaborator

@msureshkumar88 msureshkumar88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good performance direction overall. A few changes needed before this can merge.

Blocking:

  1. E2E parity test stale tool name hints — the new test will unconditionally skip in CI after the rename (see inline comment)
  2. Predictable session IDs — sequential integers are trivially enumerable; replace with a random token
  3. Unbounded session growth — no cap means the session store is an OOM vector
  4. mcp_error_response_with_status interpolates message directly into JSON without escaping — inconsistent with mcp_text_result_response which uses serde_json::to_string
  5. echo delay has no upper bound — a client can stall a handler with an arbitrarily large value

Non-blocking suggestions:

  • FastTestServer rmcp layer is never mounted — either remove the ~150-line dead path or document that it's an unreachable reference implementation
  • tools/list hardcodes a raw JSON string that duplicates the #[tool] declarations — two sources of truth that can drift
  • mcp_base_headers() allocates a new HeaderMap on every response — a LazyLock<HeaderMap> or axum tuple response would eliminate this allocation from the hot path
  • FastTestServer::request_count is never incremented in production (only in the dead rmcp path)
  • PR says "Refs #4497" — confirm whether this closes the issue or partially addresses it

Comment thread tests/live_gateway/mcp/test_fast_server_go_rust_parity.py Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Comment thread mcp-servers/rust/fast-time-server/src/main.rs Outdated
Signed-off-by: lucarlig <lucarlig@protonmail.com>
@lucarlig lucarlig requested a review from msureshkumar88 June 5, 2026 13:09
Signed-off-by: lucarlig <lucarlig@protonmail.com>
Copy link
Copy Markdown
Collaborator

@msureshkumar88 msureshkumar88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All requested changes addressed and verified against the source:

  • Session IDs now use Uuid::new_v4() — no longer enumerable
  • Session cap enforced at 10 000; initialize returns 503 when full, with a unit test covering the boundary
  • mcp_error_response_with_status uses serde_json::to_string for message escaping — JSON injection path closed
  • Echo delay capped at 60 s via validate_delay() in both the MCP and REST paths
  • Parity test hints updated to fast-time prefix — no longer silently skips in CI
  • rmcp layer removed entirely (dead code + stale deps gone)
  • mcp_json_response returns an axum tuple — HeaderMap alloc eliminated from the hot path

Good work turning these around cleanly. Ready to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fast time server go to rust

2 participants