feat: MCP server with management tools + SSE transport#28
Merged
christianromeni merged 2 commits intomainfrom Mar 27, 2026
Merged
feat: MCP server with management tools + SSE transport#28christianromeni merged 2 commits intomainfrom
christianromeni merged 2 commits intomainfrom
Conversation
Implements an MCP (Model Context Protocol) server at /api/v1/mcp/voidllm
exposing VoidLLM management capabilities as tools for IDE integration
(Claude Code, Cursor, Windsurf).
Protocol: JSON-RPC 2.0 over Streamable HTTP. No external MCP library —
implemented directly on Fiber for full control, performance, and privacy.
Tools:
- list_models — RBAC-scoped: admin sees full info (strategy, deployments),
member sees only name + type
- get_model_health — per-model/deployment health status
- get_usage — usage stats scoped to caller's key/org
- list_keys — API keys visible to caller (own for member, all for admin)
- create_key — create temporary API key with optional expiry
- list_deployments — deployment details (system_admin only)
All tools fully implemented against real DB — no stubs.
Security:
- Auth via existing Bearer token middleware
- RBAC checks per tool matching REST API permissions
- Internal errors sanitized ("internal error", no DB details leaked)
- Audit middleware excludes MCP (tool calls are not auditable mutations)
- Defensive body copy (fasthttp buffer recycling safe)
- MCP spec-compliant notification handling
Tests: 91 MCP tests — protocol, server, tool implementations, RBAC,
integration, error sanitization, concurrency.
- POST Accept: text/event-stream wraps JSON-RPC response in SSE format - GET /mcp/voidllm opens SSE stream (endpoint event + 30s keep-alive) - 10 minute stream deadline prevents DoS via connection exhaustion - Proper Accept header parsing (comma-separated, quality params) - README: MCP Integration section with setup guide + tool reference - configuration.md: MCP server docs (endpoints, tools, RBAC, privacy) - Landing page (EN + DE): MCP server in community feature list - 7 SSE transport tests (POST+GET, auth, headers, notifications)
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Model Context Protocol (MCP) server to VoidLLM for IDE integration. Claude Code, Cursor, and Windsurf can manage VoidLLM directly via MCP tools.
MCP Server
SSE Transport
Privacy
Docs
Test plan
go test ./... -race— 19 packages, 91+ MCP testsnpx tsc --noEmit && npm run lint