perf: Go benchmark CLI with Vegeta — LLM, MCP, and Code Mode#32
Merged
christianromeni merged 4 commits intomainfrom Mar 29, 2026
Merged
perf: Go benchmark CLI with Vegeta — LLM, MCP, and Code Mode#32christianromeni merged 4 commits intomainfrom
christianromeni merged 4 commits intomainfrom
Conversation
…N reports Replace bash script with a proper Go CLI using Vegeta as a library. Embedded mock servers (LLM + MCP), warmup phase, text + JSON reports. Scenarios: quick, sustained, burst, large-payload, mixed, endurance. Usage: go run ./scripts/bench [scenario] [--rps N] [--duration D] [--json]
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
clawbolt
pushed a commit
to clawbolt/voidllm
that referenced
this pull request
Apr 3, 2026
…ion (voidmind-io#30) * feat: Code Mode — WASM-sandboxed JS execution for MCP tool orchestration (voidmind-io#30) Add Code Mode: LLMs write JavaScript that orchestrates multiple MCP tool calls in a single execution, reducing token usage by 30-80%. Scripts run in a QuickJS/WASM sandbox (fastschema/qjs + Wazero) embedded in VoidLLM. Three new built-in MCP tools: - list_servers: discover available MCP servers - search_tools: find tools by keyword across servers - execute_code: run JS with MCP tools as async functions Runtime: pool of QJS runtimes (default 8), fresh runtime per execution, tool schema cache with lazy fetch, console capture in results. Configurable via voidllm.yaml (disabled by default): mcp.code_mode.enabled, pool_size, memory_limit_mb, timeout, max_tool_calls Also fixes session re-init double-check in MCP proxy. * feat: Code Mode Phase 3 — blocklist, refresh, toggle, MCP server split (voidmind-io#31) Split built-in MCP into two servers: - /api/v1/mcp — Code Mode (list_servers, search_tools, execute_code) - /api/v1/mcp/voidllm — Management (list_models, get_usage, etc.) - /api/v1/mcp/:alias — External MCP server proxy Per-tool blocklist for Code Mode: - Migration 0005: mcp_tool_blocklist table - CRUD API: GET/POST/DELETE /mcp-servers/:id/blocklist - Defense in depth: filtered before sandbox injection + checked in ToolCaller - Blocklist also applied to search_tools and list_servers tool counts Tool refresh endpoint: - POST /mcp-servers/:id/refresh-tools with 60s cooldown Admin controls: - code_mode_enabled toggle in API response and PATCH - UI: Code Mode toggle column, expanded row with blocklist management Shared MCP handler helper eliminates POST/SSE handler duplication. * feat: Code Mode Phase 4 — Proxy pattern, SSE upstream, execution history, TypeScript types (voidmind-io#32) JS Proxy pattern replaces static preamble generator: - Single __callTool dispatch via ES6 Proxy interception - Any tool name characters supported, preamble is O(1) in tool count SSE upstream transport support: - Auto-detect Streamable HTTP vs deprecated SSE protocol - Lazy detection with sync.Once, origin validation on endpoints Execution history: - Migration 0006: code_mode_execution_id on mcp_tool_calls - UUIDv7 per execute_code call groups all tool calls Dynamic TypeScript types in execute_code description: - GenerateToolTypeDefs converts cached tool schemas to TS declarations - OnToolsListHook injects types at tools/list time Bug fixes: - MCP access control enforced in all Code Mode closures for global servers - ToolCache fetcher resolves servers across all scopes - Frontend blocklist DELETE matches backend query parameter API * feat: persistent tool cache, SSE detection, tools list UI with block buttons (voidmind-io#33) Persistent tool cache: - Migration 0007: mcp_server_tools table for DB-backed tool schemas - Startup loads from DB (zero HTTP calls, TypeScript types immediately available) - 24h background refresh keeps schemas current - Write-through on every fetch (RefreshServer, GetTools) - DB entries marked stale on load so they refresh within maxAge SSE transport detection: - Servers using deprecated SSE protocol auto-deactivated at startup - Clear error message: "server uses deprecated SSE transport" - Test connection also detects and deactivates SSE servers Tools list UI: - GET /mcp-servers/:id/tools endpoint returns cached tools with blocked status - Expanded row shows all tools with Block/Unblock buttons - Block buttons work for YAML-sourced servers (blocklist is independent of source) - Plug icon centered in sidebar and MCP servers page Also fixes: - ToolStore.Delete uses server ID (not alias) to avoid soft-delete lookup failure - Corrupt JSON schemas skipped on DB load instead of serving empty schemas * ci: lower patch coverage target to 50% * refactor: extract Code Mode service from app.go + add 46 tests Extract 3 closures (ExecuteCode, ListAccessibleMCPServers, SearchMCPTools) from app.go into codeModeService in code_mode.go. Shared accessibleServers helper eliminates duplicated server-listing + access-check logic. New tests: - code_mode_test.go: 21 tests (mock DB, real WASM executor) - mcp_tool_blocklist_test.go: 11 tests (CRUD, conflicts, isolation) - mcp_server_tools_test.go: 14 tests (upsert, replace, active filter) app.go reduced by ~400 lines. * test: add 13 handler tests for blocklist, refresh, and tools list endpoints * docs: add Code Mode section to README with config, limitations, and IDE setup * test: add 17 tests for CallMCPTool and dbToolStore
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Professional benchmark tool replacing the bash script. Uses Vegeta as a Go library with embedded mock servers, warmup phase, and structured reports.
Baseline results (1000 RPS, 60s):
Scenarios
quicksustainedburstlarge-payloadmixedenduranceUsage
go run ./scripts/bench quick go run ./scripts/bench sustained --rps 2000 --duration 120s go run ./scripts/bench quick --json > results.jsonTest plan
go build ./scripts/bench/compilesquickscenario: 100% success, all 4 phases + Code Mode