perf: sonic JSON, MCP caches, transport pool, unified parse#33
Merged
christianromeni merged 8 commits intomainfrom Mar 30, 2026
Merged
perf: sonic JSON, MCP caches, transport pool, unified parse#33christianromeni merged 8 commits intomainfrom
christianromeni merged 8 commits intomainfrom
Conversation
BREAKING CHANGES: 1. ToolCache keyed by server ID instead of alias. Prevents tool schema conflicts when multiple scopes use the same alias. ToolFetcher resolves servers by ID. Persistent tool store keys by server ID. 2. MCP access for global servers is now closed-by-default at the org level. Orgs must explicitly configure org_mcp_access to grant access to global MCP servers. Team and key levels remain open (inherit from org). Built-in VoidLLM server is unaffected (always accessible). 3. Removed GetMCPServerByAliasAny (no longer used). Tests: 30 new tests for CheckMCPAccess, MCPAccessCache, and accessibleServers.
…refresh - Deferred transport closure via stale list for safe eviction - Consolidated refreshMCPCaches into single DB query for both caches - Removed unused field from resolved server struct - Added error logging for decrypt failures in cache load - Removed dead code guard in metrics path
Register bench MCP server as org-scoped via Admin API instead of relying on global YAML server. Updates MCP proxy target alias to match the org-scoped registration.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Closed-by-default MCP access requires a way for org admins to grant access to global MCP servers. Adds: - 6 API endpoints for org/team/key MCP access (GET/PUT) - Lightweight available-servers endpoint for org admins - Org MCP Access tab in Organization settings - Team MCP Access tab in Team settings - System admin bypass for MCP access checks - 14 integration tests
Test was asserting 0 servers for system_admin, but the bypass now grants unrestricted access. Split into member (denied) and system_admin (allowed) assertions.
clawbolt
pushed a commit
to clawbolt/voidllm
that referenced
this pull request
Apr 3, 2026
…ion (voidmind-io#30) * feat: Code Mode — WASM-sandboxed JS execution for MCP tool orchestration (voidmind-io#30) Add Code Mode: LLMs write JavaScript that orchestrates multiple MCP tool calls in a single execution, reducing token usage by 30-80%. Scripts run in a QuickJS/WASM sandbox (fastschema/qjs + Wazero) embedded in VoidLLM. Three new built-in MCP tools: - list_servers: discover available MCP servers - search_tools: find tools by keyword across servers - execute_code: run JS with MCP tools as async functions Runtime: pool of QJS runtimes (default 8), fresh runtime per execution, tool schema cache with lazy fetch, console capture in results. Configurable via voidllm.yaml (disabled by default): mcp.code_mode.enabled, pool_size, memory_limit_mb, timeout, max_tool_calls Also fixes session re-init double-check in MCP proxy. * feat: Code Mode Phase 3 — blocklist, refresh, toggle, MCP server split (voidmind-io#31) Split built-in MCP into two servers: - /api/v1/mcp — Code Mode (list_servers, search_tools, execute_code) - /api/v1/mcp/voidllm — Management (list_models, get_usage, etc.) - /api/v1/mcp/:alias — External MCP server proxy Per-tool blocklist for Code Mode: - Migration 0005: mcp_tool_blocklist table - CRUD API: GET/POST/DELETE /mcp-servers/:id/blocklist - Defense in depth: filtered before sandbox injection + checked in ToolCaller - Blocklist also applied to search_tools and list_servers tool counts Tool refresh endpoint: - POST /mcp-servers/:id/refresh-tools with 60s cooldown Admin controls: - code_mode_enabled toggle in API response and PATCH - UI: Code Mode toggle column, expanded row with blocklist management Shared MCP handler helper eliminates POST/SSE handler duplication. * feat: Code Mode Phase 4 — Proxy pattern, SSE upstream, execution history, TypeScript types (voidmind-io#32) JS Proxy pattern replaces static preamble generator: - Single __callTool dispatch via ES6 Proxy interception - Any tool name characters supported, preamble is O(1) in tool count SSE upstream transport support: - Auto-detect Streamable HTTP vs deprecated SSE protocol - Lazy detection with sync.Once, origin validation on endpoints Execution history: - Migration 0006: code_mode_execution_id on mcp_tool_calls - UUIDv7 per execute_code call groups all tool calls Dynamic TypeScript types in execute_code description: - GenerateToolTypeDefs converts cached tool schemas to TS declarations - OnToolsListHook injects types at tools/list time Bug fixes: - MCP access control enforced in all Code Mode closures for global servers - ToolCache fetcher resolves servers across all scopes - Frontend blocklist DELETE matches backend query parameter API * feat: persistent tool cache, SSE detection, tools list UI with block buttons (voidmind-io#33) Persistent tool cache: - Migration 0007: mcp_server_tools table for DB-backed tool schemas - Startup loads from DB (zero HTTP calls, TypeScript types immediately available) - 24h background refresh keeps schemas current - Write-through on every fetch (RefreshServer, GetTools) - DB entries marked stale on load so they refresh within maxAge SSE transport detection: - Servers using deprecated SSE protocol auto-deactivated at startup - Clear error message: "server uses deprecated SSE transport" - Test connection also detects and deactivates SSE servers Tools list UI: - GET /mcp-servers/:id/tools endpoint returns cached tools with blocked status - Expanded row shows all tools with Block/Unblock buttons - Block buttons work for YAML-sourced servers (blocklist is independent of source) - Plug icon centered in sidebar and MCP servers page Also fixes: - ToolStore.Delete uses server ID (not alias) to avoid soft-delete lookup failure - Corrupt JSON schemas skipped on DB load instead of serving empty schemas * ci: lower patch coverage target to 50% * refactor: extract Code Mode service from app.go + add 46 tests Extract 3 closures (ExecuteCode, ListAccessibleMCPServers, SearchMCPTools) from app.go into codeModeService in code_mode.go. Shared accessibleServers helper eliminates duplicated server-listing + access-check logic. New tests: - code_mode_test.go: 21 tests (mock DB, real WASM executor) - mcp_tool_blocklist_test.go: 11 tests (CRUD, conflicts, isolation) - mcp_server_tools_test.go: 14 tests (upsert, replace, active filter) app.go reduced by ~400 lines. * test: add 13 handler tests for blocklist, refresh, and tools list endpoints * docs: add Code Mode section to README with config, limitations, and IDE setup * test: add 17 tests for CallMCPTool and dbToolStore
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Performance optimizations for the MCP proxy hot path, reducing overhead from ~670µs to ~420µs P50 at 1000 RPS.
internal/jsonx— drop-in wrapper with ConfigStd for encoding/json compatibility. All 17 production files migrated.parseMCPRequestMetareplaces 3 separate JSON parses per request.Breaking Changes
org_mcp_access. Org-scoped and team-scoped servers are unaffected.Benchmark Results (1000 RPS, 30s)
Test plan
go test ./... -race -count=1— all pass, no race conditionsgo vet ./...— cleango run ./scripts/bench quick --rps 1000 --duration 30s— 100% success all paths