feat: explorer mcp server for scenes#9339
Open
pravusjif wants to merge 73 commits into
Open
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t/explorer-mcp-server
Contributor
|
Windows and Mac build successful in Unity Cloud! You can find a link to the downloadable artifact below. |
pravusjif
added
force-build
Contributor
|
Warnings not reduced: 14668 => 14766 — remove at least one warning to merge. Warnings/errors in files changed by this PR (97)
…and 47 more (see the |
…eat/explorer-mcp-server Resolved conflict in DynamicWorldContainer.cs: dropped the unused 'using CommunicationData.URLHelpers;' brought by the remote side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs
decentraland-bot
self-requested a review
July 17, 2026 15:45
decentraland-bot
left a comment
decentraland-bot
left a comment
Contributor
There was a problem hiding this comment.
STEP 1 — Scope
New DCL.McpServer assembly (Core, Tools, Components, Systems, Utils, Tests) plus targeted changes to CinemachineExtensions, ApplyCinemachineCameraInputSystem, DynamicWorldContainer, FeaturesRegistry, AppArgsFlags, DiagnosticsContainer, and ReportCategory. Reviewed against CLAUDE.md, docs/code-style-guidelines.md, and the review-instructions prompt. Repository cloned and searched with rg for lifecycle owners, teardown mirrors, and existing conventions.
STEP 2 — Root-cause check
This is a new feature (embedded MCP server for coding-agent scene automation), not a bug fix. The diff directly builds the stated capability. No symptom-masking, swallowed exceptions, or workaround patterns detected. PASS.
STEP 3 — Design & integration
Owner search:
McpServerPluginis a newIDCLGlobalPluginWithoutSettingsregistered inDynamicWorldContainer.CreateAsyncbehindFeatureId.MCP_SERVER. This is the correct registration point for a global feature plugin — confirmed by searching the ~50 otherglobalPlugins.Add(...)calls in the same file.McpInputOverrideSystemruns inInputGroupafterUpdateInputMovementSystem/UpdateInputJumpSystem. The movement input lifecycle is owned by these systems; the MCP override correctly injects into the same pipeline via a transient intent component.McpPointerClickSystemruns inPresentationSystemGroupafterProcessPointerEventsSystem. The pointer-event lifecycle is owned byProcessPointerEventsSystem; the MCP system injects a synthetic press through the sameAppendPointerEventResultsIntentpath, which is the correct seam.- No duplicate lifecycle management found. The intent components (
McpMovementOverride,McpPointerClickIntent) are transient ECS components written by tools and consumed/removed by the systems — standard ECS intent pattern.
Teardown trace:
McpServerPlugin.Dispose()correctly disposesscreenshotTool,server, andserverCts.⚠️ TheDebugMenuConsoleLogEntryBusevent subscription (logEntryBus.MessageAdded += logBuffer.Append, line 98) has no teardown path —logEntryBusis a local variable never stored as a field. This matches the identical pattern inDebugMenuPlugin(confirmed: same constructor pattern, same emptyDispose()), and both are process-lifetime plugins. No real-world leak, but the subscription can never be removed. P2 — store as field for future cleanup parity.
Design verdict: PASS — no misplaced logic, no duplicate lifecycle, no per-frame reconciliation of explicit moments.
STEP 4 — Member audit
| Member | Consumers | Verdict |
|---|---|---|
SetCameraModeTool.IsModeChangeAllowed() |
GetPlayerStateTool (camera.modeChangeAllowed), TrySwitchMode (internal) |
2 consumers — justified |
SetCameraModeTool.TrySwitchMode() |
SetCameraModeTool.ExecuteAsync, SetCameraPoseTool.ExecuteAsync |
2 consumers — justified |
CinemachineExtensions.ForceFreeCameraPose() |
SetCameraPoseTool.ExecuteAsync |
1 consumer — follows the established Force*CameraLookAt pattern on ICinemachinePreset; acceptable as a cohesive extension group |
CinemachineExtensions.ForceFreeCameraLookAt() |
ApplyCinemachineCameraInputSystem.ForceLookAt (Free case) |
1 consumer — fixes a bug where Free mode fell through to ThirdPerson look-at logic |
McpToolAnnotations.ReadOnly() / .Mutating() |
All 16 tools | Multi-consumer |
No single-use predicates re-checking caller-guaranteed invariants, no absent≠false conflation, no re-derived lookups found.
STEP 5 — Line-level findings
All findings are posted as inline comments with suggestion blocks below.
Summary of findings
| # | Severity | File | Issue |
|---|---|---|---|
| 1 | P1 | McpHttpServer.cs:196 |
Method name typo WriteEmptyAncClose → WriteEmptyAndClose (naming standard) |
| 2 | P1 | Utils/SceneLogBuffer.cs:4, Utils/JObjectExtensions.cs:6 |
Namespace DCL.McpServer.Tools doesn't match Utils/ folder — CLAUDE.md forbids suppressing CheckNamespace |
| 3 | P2 | McpHttpServer.cs:36 |
Double semicolon |
| 4 | P2 | McpHttpServer.cs:46-47 |
Redundant ?. after null guard |
| 5 | P2 | McpHttpServer.cs:128 |
ContentLength64 check bypassed by chunked encoding (-1 > MAX is false) |
| 6 | P2 | McpPointerClickSystem.cs:170 |
Inline epsilon 0.0001f — extract to named constant |
| 7 | P2 | McpServerPlugin.cs:97-98 |
logEntryBus not stored as field — event subscription has no teardown path |
Security assessment
- Localhost binding (
127.0.0.1only) — correct, never network-reachable. - Origin validation — rejects browser-originated cross-origin requests; CLI tools (no
Originheader) are allowed by design. Sufficient for a localhost dev tool. - No authentication token — acknowledged deferral in the PR. The server is opt-in (
--mcpflag), localhost-only, and targets local development workflows. Acceptable for v1; token should land before any remote-reachable or multi-user scenario. - Input validation — all tool parameters are clamped/validated.
SendChatToolcaps message length at 500 chars. No injection vectors found. - Thread safety — every tool that touches ECS/Unity correctly switches to main thread.
SceneLogBufferuseslockfor cross-thread access.ScreenshotToolusesInterlockedcapture gate. No unsynchronized shared mutable state.
STEP 6 — Complexity
COMPLEX — introduces new ECS systems, components, plugin registration, assembly definitions, and input handling integration.
STEP 7 — QA assessment
QA_REQUIRED: YES — modifies runtime code, adds input simulation systems, camera control, and scene interaction tools.
STEP 8 — Non-blocking warnings
No warnings triggered (Main.unity not modified).
STEP 9 — Verdict
REVIEW_RESULT: PASS ✅
COMPLEXITY: COMPLEX
COMPLEXITY_REASON: Introduces new ECS systems (McpInputOverrideSystem, McpPointerClickSystem), components, a new assembly (DCL.McpServer), plugin registration, HTTP server, and input/camera handling integration.
QA_REQUIRED: YES
Reviewed by Jarvis 🤖 · Requested by decentraland-bot via GitHub
Ludmilafantaniella
approved these changes
Jul 17, 2026
Ludmilafantaniella
left a comment
Ludmilafantaniella
left a comment
Contributor
There was a problem hiding this comment.
Tested on Windows and Mac. ✅
Used the sdk7-scene-template as the local scene. Connected Claude Code to the Explorer MCP server and prompted it to build a labyrinth game from scratch without restarting the scene.
Windows:
- MCP server connected successfully
- Claude Code built the labyrinth, verified collisions and geometry via screenshots and locomotion
- Scene logs confirmed:
MAZE: entered at point A, game startedandMAZE: exited at point B, WON in 141.6s✅
Mac:
- MCP server connected (required
--scope projectinstead of--scope userdue to permissions) - Claude Code built and verified the labyrinth
- Game completed successfully:
You escaped the labyrinth! Time: 107.0svisible on screen ✅
In both cases the scene was never restarted and scene.json was untouched throughout. LGTM 🚀
✅Smoke test performed:
- ✔️ Backpack and wearables in world
- ✔️ Emotes in world and in backpack
- ✔️ Teleport with map/coordinates/Jump In
- ✔️ Chat and multiplayer
- ✔️ Camera
- ✔️ Skybox
# Conflicts: # Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset
…stion The Content-Length guard in HandlePostAsync was bypassed by Transfer-Encoding: chunked requests (ContentLength64 == -1), letting ReadToEndAsync read an unbounded body into memory. Read into a fixed MAX_BODY_BYTES buffer and reject oversized bodies with 413.
…s teardown
- McpHttpServer: cap POST body read into a fixed buffer so the 1MB limit
can't be bypassed via Transfer-Encoding: chunked (ContentLength64 == -1)
- McpServerPlugin: store DebugMenuConsoleLogEntryBus as a field and
unsubscribe logBuffer.Append in Dispose()
- McpHttpServer: cap POST body read via ReadBlockAsync so chunked requests
can't bypass the Content-Length limit and exhaust memory
- McpHttpServer: capture the listener into a local in the accept loop to
avoid an NRE race with a concurrent Dispose()
- McpHttpServer: document the deliberate stateless single-session design
(unconditional, unvalidated Mcp-Session-Id)
- McpToolsRegistry: hand out a DeepClone of the tools/list payload so
concurrent dispatches don't race on the shared JObject
- McpJsonRpcDispatcher: return -32600 (Invalid Request) for well-formed
JSON that isn't an object, keeping -32700 for unparseable input
- McpJsonRpcDispatcher: log a warning when the client's initialize
protocolVersion differs from the server's
- McpServerPlugin: store the log entry bus in a field and unsubscribe in
Dispose()
…eat/explorer-mcp-server
…hed payload The implicit JObject conversion on McpToolsRegistry hid a DeepClone behind `Result(id, tools)` in the dispatcher — a surprising conversion with a per-request allocation. Serialize the tools/list payload once in Build() and hand it out via an explicit ToolsListPayload() that wraps the immutable JSON string in a fresh JRaw, so concurrent dispatches share it without cloning or re-parenting a JToken tree. - Drop `implicit operator JObject` and the per-request DeepClone - Store the sealed payload as a serialized string, built once in Build() - Update the dispatcher call site to tools.ToolsListPayload() - Re-parse the raw payload in registry tests via a Payload() helper
Replace repeated routable.Value.X access with a single tuple deconstruction and a property-pattern guard in DispatchAsync.
Centralize Formatting.Indented serialization in McpToolResult so tools can't drift the structured payload from its text mirror or forget the formatting. Convert the eight callsites and drop their now-unused Newtonsoft.Json import.
The builder produces both input and output JSON Schemas (OutputSchema, VectorSchema), so the "Input" name misrepresented it. Pure rename of the type and its file/test.
The three structured tools declare OutputSchema and build the payload by hand, so a field added to one and forgotten in the other would silently misdescribe the result. Add McpSchemaAssert.KeysMatch, a recursive schema-vs-payload key check, and wire it into the Get*/List* tool tests (populating a camera entity and a scene facade so nested objects are covered too).
… Build Dedup nullable type-token logic in McpJsonSchema via a private TypeToken(type, nullable) helper, dropping the (JToken)type cast in Property and sharing it with Object. Add symmetric OutputSchema validation in McpToolsRegistry.Build so a malformed output schema fails fast at registration instead of reaching the client.
Change the extension to take HttpStatusCode instead of int, casting once inside the method. Drops the (int) cast and statusCode: label from all seven call sites and makes the status intent type-safe.
Cap each tools/call at TOOL_CALL_TIMEOUT with a CancellationTokenSource linked to the server-lifetime token. A hung tool now returns a timeout error result instead of holding its HttpListenerContext open until shutdown. Distinguish shutdown cancellation (rethrow) from timeout (error result) via a when (ct.IsCancellationRequested) filter.
Tools declared "switch to the main thread yourself" and 15 of 16 opened with an identical `await UniTask.SwitchToMainThread(ct)`. Hop once in McpJsonRpcDispatcher.CallToolAsync instead and flip the IMcpTool contract to "invoked on the main thread". Readers with no other await become non-async (UniTask.FromResult); tools that offload to the thread pool keep their return-trip hops.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Embeds an MCP (Model Context Protocol) server inside the Explorer so a coding agent (Claude Code, Codex, …) can see a running SDK7 scene (screenshot, player/scene state, scene console logs) and drive it (teleport, move, walk, look, camera, pointer clicks, chat, emotes, reload) — closing the edit → reload → verify loop for scene development without a human in the middle.
The primary target is a retail build launched by Creator Hub (no Unity Editor, no Unity license on the creator's machine); the same server runs unchanged in the Editor for internal dev and on PR builds for QA.
Scope & blast radius
--mcp/--mcp-port <port>(accepted from the command line or a deep link). Default port 8123.FeatureId.MCP_SERVER(FeaturesRegistry, resolved asappArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT));DynamicWorldContainer.CreateAsyncaddsMcpServerPluginonly when the feature is enabled.HttpListenerbinds to127.0.0.1and is never reachable from the network.DCL.McpServerassembly + two folded folders); no#if UNITY_EDITOR— Editor and build behave identically.Architecture (brief)
DCL.McpServerasmdef rooted atExplorer/Assets/DCL/McpServer/, split intoCore/(protocol + transport + tool contract),Tools/(one class per tool),Components/,Systems/,Utils/,Tests/.Systems/folds intoDCL.PluginsandTests/intoDCL.EditMode.Testsvia.asmrefso they can reach code the standalone assembly doesn't reference.System.Net.HttpListener(Streamable-HTTP), Newtonsoft, protocol spec2025-06-18,toolscapability only.McpHttpServer(server + Origin validation) →McpJsonRpcDispatcher(initialize/ping/tools/list/tools/call) →IMcpTool.McpMovementOverride,McpPointerClickIntent) + aUniTaskCompletionSource, consumed and removed by aBaseUnityLoopSystem(McpInputOverrideSystemin the input group,McpPointerClickSystemin presentation) so synthetic input/clicks flow through the real pipelines. Each toolawaitsSwitchToMainThreaditself before reading ECS/Unity; heavy work (base64) hops back to the pool.Key decisions & rationale
Standalone embedded server — no integration with Unity's official MCP.
Unity's official MCP (in
com.unity.ai.assistant) is Editor-only by construction: its transport chain runs through a relay binary into the Unity Editor process, with no runtime component for a built player, and its extension API is discovered by Editor-startupTypeCachescanning that doesn't exist in a player. It is also license-gated (Unity 6+, Assistant package, AI-Gateway sign-in, a per-tier connection entitlement) — creators have none of these — and its ~51 tools target Editor asset/script/shader work with no input simulation and no built-player interaction at all. For our primary scenario (retail build, no Editor) integration is simply impossible. Owning the server is cheap because the MCP layer is a thin protocol adapter; all real capability (back-buffer capture, input injection, ECS reads) must live in-process regardless of who serves the protocol.Embedded MCP-over-HTTP because Creator Hub will drive multiple engines, not just Unity.
The product direction is a Creator Hub gateway: the coding agent talks to one endpoint (Creator Hub), which fans out as an MCP client to whichever engine's embedded server is running — Unity, Bevy, or Godot. For Hub to use one generic MCP client across all of them, every engine must expose the same MCP-over-HTTP surface. That requirement is what drove the transport choice and what set aside the parallel automation approaches:
ws://1473, Fleck) is Unity/web-specific — Bevy and Godot will never ship it, so routing Unity through CDP would force Hub to maintain a bespoke non-MCP client just for Unity. It stays a sibling network-monitor concern, not the automation transport.DclHarness/ClaudeIPCprototype is an Editor-only, reflection-based file channel bound to a running Unity Editor — no retail/multi-engine path at all.An embedded, standalone MCP server per engine is precisely what makes the multi-engine gateway possible.
Hand-rolled JSON-RPC — not the
ModelContextProtocolC# SDK; no batching.The SDK's most valuable piece (HTTP hosting) lives in
ModelContextProtocol.AspNetCore(Kestrel / Generic Host), which Unity doesn't run — we'd still have to write our ownHttpListenertransport and the main-thread marshalling. Its reflection/assembly-scan tool discovery and runtime schema generation are exactly the patterns that get fragile under IL2CPP / managed-code stripping in retail builds, whereas our explicit registration is IL2CPP-safe by construction. It would also drag inSystem.Text.Json+Microsoft.Extensions.*, conflicting with the codebase's Newtonsoft + own composition-root DI. We borrow the SDK's design (schema-from-signature, DI-injected params, annotation hints), not its code — the whole dispatcher is ~150 lines. We track spec 2025-06-18, which removed JSON-RPC batching, so a single-object dispatcher (no batch-array path) is spec-correct, not a gap.Tool annotations (
readOnlyHint/destructiveHint/idempotentHint/openWorldHint, spec 2025-06-18) are emitted intools/list— cheap, forward-compatible, and the machine-readable hook a future auth-gate of mutating tools (and profile filtering) will read.Structured output seam (
structuredContent+outputSchema) is built as a purely-additive result envelope and adopted only as an exemplar on the read-only state tools (get_player_state,get_scene_state,list_scene_entities) that already build a JObject; every other tool returns text only. Doing it now locks the output-field contract while zero external consumers depend on the prose format.Ecosystem sanity-check. A sweep of third-party Unity MCP servers + Anthropic's tool-design guidance confirmed the foundation is spec-aligned. Almost the entire ecosystem is Editor-tooling with a sidecar process; being fully in-process inside a running player is unusual and correct — it sidesteps the domain-reload churn that is the ecosystem's dominant pain by construction. Curated ~16 workflow-shaped tools avoid the tool-list flooding that degrades LLM tool selection.
Security model
127.0.0.1only.Originis localhost (drive-by / DNS-rebinding defense); requests without anOriginheader (CLI clients, curl) are allowed.execute-arbitrary-C#tool (a foot-gun for a retail build with no auth). The trigger to add a token is the first mutating dev-only tool.Tool catalog (16 tools)
Full arguments/returns and troubleshooting in
docs/mcp-automation.md.screenshot,get_player_state,get_scene_state,get_scene_logs,list_scene_entities,get_entity_details.teleport,move_to,walk,look_at,set_camera_mode,set_camera_pose,send_chat,reload_scene,trigger_emote,click_entity.Tests
EditMode tests (folded into
DCL.EditMode.Tests) cover the load-bearing routing plus the input bridge:McpJsonRpcDispatcherShould— known tool dispatch, unknown tool →-32602, notification (id == null) dropped.McpToolsRegistryShould—tools/listserialization,TryGet/ name-guard.McpToolResultShould— result / error / structured-output envelope.McpHttpServerShould,McpInputSchemaShould.McpPointerClickSystemShouldand per-tool tests (GetEntityDetailsToolShould,ListSceneEntitiesToolShould,GetPlayerStateToolShould,GetSceneStateToolShould).Deliberately NOT in this PR / follow-ups
Iteration-1 holes are intentional — the foundation is built to extend organically:
notifications/cancelledhandler + a per-request CTS registry land with the first genuinely long-running/interruptible tool.get_entity_details(char cap) andlist_scene_entities(limit + "narrow with X"); acomponent-name filter needs extendingIWorldInfo(out of scope).tools-only (noresourcescapability declared): read-only state that is spec-naturally a Resource (scene state, entity list, log buffer) stays a polling tool for now. Two narrow slices are worth adopting later —resource_linkin results (selectively; not forscreenshot, which agents consume best as an inline image) andresources/subscribe(the one real win over polling, adopted only when a consumer needs push and the target client supports it).verb_nounsnake_case; the only engine-specific risk isset_camera_modevaluesdrone/free, to be isolated behind a profile later).--mcp-profile creator|dev, auth token, per-tool toggles — designed, not built.Related: V2 issue #9393
How to enable / test
Endpoint:
http://127.0.0.1:<port>/unity-explorer-mcp. Connect an agent:Connection details:
docs/mcp-automation.md#connecting-a-coding-agent.Test Steps
[simpler SKILL pending for creators to use the MCP server in a more efficient way]
positionvalue):"C:\Users\[YOUR-USER]\Downloads\Decentraland_windows64\Decentraland.exe" --realm http://127.0.0.1:8000 --position 0,0 --local-scene true --skip-version-check true --mcpopen Decentraland.app --args --realm http://127.0.0.1:8000 --position 0,0 --local-scene true --skip-version-check true --mcpclaude mcp add --transport http --scope user explorer http://127.0.0.1:8123/mcp.