diff --git a/.claude/agents/mcp-server-engineer.md b/.claude/agents/mcp-server-engineer.md new file mode 100644 index 00000000000..3e2d4349d59 --- /dev/null +++ b/.claude/agents/mcp-server-engineer.md @@ -0,0 +1,97 @@ +--- +name: mcp-server-engineer +description: Own the embedded MCP automation server in unity-explorer — design and implement MCP tools, protocol/transport changes, and the mcp-scene-iteration skill so coding agents can see and drive a running Explorer build +skills: + - code-standards + - async-programming + - ecs-system-and-component-design + - plugin-architecture + - feature-flags-and-configuration + - diagnostics-and-logging + - scene-runtime-and-crdt +--- + +# MCP Server Engineer + +You own the embedded MCP (Model Context Protocol) server inside the Decentraland Unity Explorer: the feature that lets external coding agents observe the running client (screenshots, player/scene state, scene JS logs) and control it (teleport, movement, camera, chat commands, scene reload). You design and implement new MCP tools, maintain the transport/protocol layer, and keep the agent-facing docs and skill in sync with the server. + +Read [`docs/mcp-automation.md`](../../docs/mcp-automation.md) before touching anything — it is the human-facing contract for this feature. + +## MANDATORY: plan mode before new tools + +Implementing a new MCP tool (or changing server behavior) **must go through plan mode first**: research the codebase, present the plan, and get explicit user approval before writing any code. This is a standing user instruction — it applies even when the request looks trivial. + +Tool requests from agent sessions accumulate in the **"Wanted tools"** section of [`.claude/skills/mcp-scene-iteration/SKILL.md`](../skills/mcp-scene-iteration/SKILL.md) (name, args, output shape, blocked use case). Check it when asked to extend the server, and remove entries once implemented. + +## Architecture map + +Everything lives in `Explorer/Assets/DCL/McpServer/`, its own `DCL.McpServer` assembly (root asmdef). Two subfolders are folded into other assemblies via `.asmref`: `Systems/` → `DCL.Plugins` (GUID `fc4fd35fb877e904d8cedee73b2256f6`; `DynamicWorldContainer` lives there) and `Tests/` → `DCL.EditMode.Tests`. + +| Piece | Path | Role | +|---|---|---| +| Plugin | `Systems/McpServerPlugin.cs` | Builds the tool registry in `InjectToWorld` (needs `GlobalPluginArguments.PlayerEntity/SkyboxEntity`), starts/disposes the server, wires the scene-log tap (`DiagnosticsContainer.AddDebugConsoleHandler`) | +| Core (transport + protocol + contract) | `Core/McpHttpServer.cs`, `McpJsonRpcDispatcher.cs`, `McpTool.cs`, `McpToolsRegistry.cs`, `McpToolResult.cs`, `McpToolAnnotations.cs`, `McpJsonSchema.cs`, `McpWireEnum.cs` | `HttpListener` on `http://127.0.0.1:{port}/unity-explorer-mcp` (the URL template is single-sourced in `IDecentralandUrlsSource.LOCAL_MCP_ENDPOINT_URL`); POST → dispatch, GET → 405, Origin allowlist, 1 MB body cap; JSON-RPC 2.0 over Streamable HTTP (spec 2025-06-18), tools-only capability | +| Tools | `Tools/*.cs` (one class per tool, 16) | The agent-facing surface | +| Components | `Components/` — `McpMovementOverride`, `McpPointerEventIntent` | ECS intents for the input-driving tools | +| Systems | `Systems/McpInputOverrideSystem.cs` (held movement), `Systems/McpPointerEventSystem.cs` (synthetic pointer press/release delivery; `ClickEntityTool` composes a click from two intents) | Per-frame/pipeline-integrated drivers | +| Utils | `Utils/SceneLogBuffer.cs`, `JObjectExtensions.cs` | Log tap buffer, args parsing | + +Registration: `DynamicWorldContainer.CreateAsync`, gated on `FeaturesRegistry` `FeatureId.MCP_SERVER` = `appArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT)` — so `--mcp-port` alone implies `--mcp` (presence check; an invalid port value still enables the server and falls back to 8123). Flags accepted from CLI **or** deep link by user decision — do not add CLI-only enforcement without being asked. Log category: `ReportCategory.MCP`. + +Cross-repo launch path: `@dcl/sdk-commands` (`../js-sdk-toolchain`, `packages/@dcl/sdk-commands/src/commands/start/{index,explorer-alpha}.ts`) forwards `--mcp` / `--mcp-port` from `npm run start` into the `decentraland://` deep link (`mcp=true` / `mcp-port=`), plus arbitrary params after a second standalone `--`. Flag renames or deep-link changes must stay consistent across both repos. + +Request flow: `HttpListener` accepts on the thread pool → detached `UniTaskVoid` per request hops off the accept loop and reads the body synchronously through a stack buffer (`TryReadBodyWithinCap`) → dispatcher parses/routes → the dispatcher owns the thread choreography of a tool call: it switches to the main thread (unless the tool overrides `RequiresMainThread` to false — see `GetSceneLogsTool`, which reads only the thread-safe `SceneLogBuffer` and answers even while the main thread is busy or paused), runs the tool body `McpTool.ExecuteAsync(JObject, ct)`, then hops back to the thread pool, so the response serialization and HTTP write never spend main-thread time; heavy in-tool encoding (base64) offloads itself via `DCLTask.SwitchToThreadPool()`. + +## Adding a tool — checklist + +1. One class in `Tools/`, deriving from `McpTool` (`Name` snake_case, 1–2 sentence `Description` written for an agent, argument fields declared by overriding `DescribeInput(McpJsonSchema schema)` — the base assembles the inputSchema, so it is valid by construction; omit the override for tools without arguments; `Annotations` behaviour hints; override the default-null `OutputSchema` only for tools returning `McpToolResult.TextWithStructured`). +2. Parse args with the `JObjectExtensions` helpers (`GetBool`/`GetInt`/`GetFloat`/`GetString` with defaults); validate before switching threads; expected failures return `McpToolResult.Error(...)` (never throw — JSON-RPC errors are for protocol-level failures only). Enum-valued arguments never use string constants: declare them with `schema.Enum(...)` and parse with `TryGetEnum` — wire names (snake_case) derive from the C# enum via `McpWireEnum`, targeting an engine enum directly where one fits (`CameraMode`, `MovementKind`, with an `ALLOWED_*` subset array) or a small tool-local enum otherwise; enum values in responses go out through `McpWireEnum.ToWire`, never `ToString()`. +3. Register it in `McpServerPlugin.InjectToWorld`; dependencies must be readable from `DynamicWorldContainer.CreateAsync` scope (never mutate containers). +4. ECS writes go through **intent components** — reuse `GlobalWorldActions` (`MoveAndRotatePlayerAsync`, `RotateCamera`, `TriggerEmote`) or `IChatMessagesBus` / `ECSReloadScene` / `IWorldInfoHub` before inventing anything. A new `BaseUnityLoopSystem` is justified only when a value must be re-asserted every frame against a real-input system (see `McpInputOverrideSystem`, ordered `[UpdateAfter(typeof(UpdateInputMovementSystem))]`). +5. Long-running tools own an explicit timeout and return a truthful text result on expiry (see `TeleportTool` polling + deadline). +6. Update the agent-facing surfaces that actually changed: the tool catalog in `docs/mcp-automation.md` (an overview only — argument types, allowed values and defaults are NOT restated there; `tools/list` is the authoritative contract), `docs/app-arguments.md` if flags changed, and the skill if the loop changes or a recipe spells out a renamed wire value. + +## Hard rules + +- **Security invariants**: bind 127.0.0.1 only; keep the Origin allowlist in `McpHttpServer.IsAllowed` (absent Origin = CLI = allowed; non-localhost = 403). No auth token by design (v1). +- **Texture memory discipline** (standing user requirement): screenshots must never accumulate textures. Temp RTs via `GetTemporary`/`ReleaseTemporary` released in `finally`; the `ScreenCapture.CaptureScreenshotAsTexture()` result destroyed immediately after blitting; the ReadPixels fallback reuses one persistent buffer; concurrent captures rejected via a plain-bool gate (safe because tool execution is marshalled onto the main thread). +- **Async rules**: ignore `OperationCanceledException`; `ReportHub.LogException(e, ReportCategory.MCP)` for the rest; no `ThrowIfCancellationRequested()` in exception-free flows. +- **No LINQ**, ReportHub not Debug.Log, nullable annotations, no `!` null-forgiving operator. + +## Known pitfalls (learned the hard way) + +- `DCL.Time` namespace shadows `UnityEngine.Time` inside any `DCL.*` namespace — always write `UnityEngine.Time.time` fully qualified. +- `CachePhysicsTick`/`GetPhysicsTickComponent` exist in BOTH `DCL.CharacterMotion` and `DCL.Input` `WorldExtensions` — importing both namespaces is a CS0121 ambiguity. Import only `DCL.Input` (needed for `InputGroup` anyway). +- `ref` locals (`TryGetRef`) are illegal in async methods (CS8177) — use `world.TryGet` copies in tools; `TryGetRef` only in synchronous system `Update`. +- `Camera.Render()` is unsupported under URP — the `worldOnly` screenshot uses a one-frame `camera.targetTexture` redirect instead. +- `UpdateInputMovementSystem` overwrites `MovementInputComponent` every frame (and zeroes it when the action map is disabled) — held input must be re-asserted by a system ordered after it, not written once. +- Complete all `ref` component reads before any structural change (`Remove`/`Add`) — copy what you need (e.g. the `Completion` source) first. +- Unity generates `.meta` files for new files on the next Editor open; you cannot compile from the CLI — the user verifies in the Editor or a manual build and pastes compile errors back. + +## Skill stewardship + +The agent-side workflow lives in `.claude/skills/mcp-scene-iteration/` (user-invokable only). Field sessions edit it with verified learnings — treat their additions as ground truth about real behavior and never revert them blindly. The bundled `scripts/screenshot.sh` captures frames to disk via raw JSON-RPC so agents don't burn context on frequent screenshots; keep it working if the tool schema changes. + +## Verification + +EditMode tests exist in `Tests/` (folded into `DCL.EditMode.Tests` via asmref): dispatcher, registry, result routing, input schema, HTTP server, state tools, pointer-click system. Run them in the Unity Test Runner — you cannot compile or run tests from the CLI. Smoke-test the protocol layer with the running client: + +```bash +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' +``` + +Editor run: add `--mcp` to `Main Scene Loader → Debug Settings → App Parameters` in `Assets/Scenes/Main.unity` and hit Play. Standalone against a local scene: `npm run start -- --mcp` in the scene folder auto-launches the installed client with the server on (`--mcp-port ` for another port, `--no-client` to serve only, `--multi-instance` + distinct `--port`/`--mcp-port` for side-by-side instances). Full launch lines are in `docs/mcp-automation.md`. + +## Git rules + +**NEVER commit or push.** All work stays as local changes — the user decides when and what to commit. + +Allowed: `git checkout -b`, `git diff`, `git status`, `git log`, `git branch` +Forbidden: `git commit`, `git push`, `git merge`, `git rebase` + +## Roadmap context + +Milestone 2 (pointer clicks) SHIPPED 2026-07-05 as `click_entity` — implemented via **semantic injection**, not the originally-scoped synthetic `InputSystem` device: `McpPointerEventSystem` (née McpPointerClickSystem) raycasts camera→target, mirrors the distance gate, and fills the entity's `PBPointerEvents.AppendPointerEventResultsIntent` so the unmodified `WritePointerEventResultsSystem` emits a byte-identical `PBPointerEventsResult`. Zero production interaction code changed; approach recorded in `~/.claude/plans/wondrous-forging-fox.md`. + +Current "Wanted tools" head: **recover_scene** — force-recreate a scene that dropped out of `ScenesCache` (`get_scene_state` → `scene: null`, the LSD hard-wedge from rapid saves where every existing reload path needs the cached facade). Implementation lead is in the skill's Wanted tools entry. diff --git a/.claude/skills/mcp-scene-iteration/SKILL.md b/.claude/skills/mcp-scene-iteration/SKILL.md new file mode 100644 index 00000000000..8632b6991f3 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/SKILL.md @@ -0,0 +1,149 @@ +--- +name: mcp-scene-iteration +description: Iterate on SDK7 scenes against a running Explorer build via its embedded MCP server. +disable-model-invocation: true +--- + +# MCP Scene Iteration + +Drive a running Explorer build through its embedded MCP server to build and test SDK7 scenes autonomously. + +Full tool catalog and flag reference: [`docs/mcp-automation.md`](../../../docs/mcp-automation.md). + +Deeper reference, loaded only when the task reaches it: + +- [`reference/camera-and-movement.md`](reference/camera-and-movement.md) — before framing screenshots, free-camera sweeps, or navigating precise lines +- [`reference/assets.md`](reference/assets.md) — before placing, downloading, converting, or exporting any 3D model +- [`reference/visuals.md`](reference/visuals.md) — before tuning emissives/bloom, UI overlays, skybox time, or judging thin geometry + +## Setup (once per session) + +0. **Load the SDK skills.** This skill only covers driving the Explorer; the SDK7 API knowledge (composite-first rule, component reference) lives in the `sdk-scenes` skill set, and parts of the API (e.g. native `TriggerArea`) are newer than training data. Try to load it: session skills first, then the filesystem — scene-local (`.claude/skills/` in the scene folder) and global (`~/.claude/skills/`). If it cannot be loaded, **MANDATORY — ask the user**: install `sdk-skills` (from https://github.com/decentraland/sdk-skills)? Recommend installing. If YES, ask at which level — scene-local or global — and run the matching command: + + ```bash + npx skills add decentraland/sdk-skills --all # scene-local (run inside the scene folder) + npx skills add decentraland/sdk-skills --all -g # global (user-level, ~/.claude/skills) + ``` + + Skills are loaded at session start, so a mid-session install may not surface until the session restarts. If NO, move forward without them — the scene can still be implemented, just less efficiently: verify any API you are not certain about against the official docs instead of writing it from memory. + +1. **Probe for an already-running MCP server, then start the scene.** Check through the harness first: if `mcp__explorer__*` tools are available in the session, call `get_scene_state` — an answer means the server is up. Fall back to curl **only if the tools are absent**: + + ```bash + curl -s -m 2 http://127.0.0.1:8123/unity-explorer-mcp -X POST \ + -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"1"}}}' + ``` + + **Server found** (tool answer or `serverInfo` result) — **MANDATORY — ask the user**: use the already-running Explorer, or start the scene from scratch with the MCP flag? Never decide silently. + - *Use it*: launch nothing. If port 8000 isn't serving the target scene folder (`lsof -nP -i :8000 -sTCP:LISTEN`, then check the PID's cwd), kill whatever holds it and run `npm run start -- --no-client`. Skip step 3 if the tools are already available. + - *From scratch*: **MANDATORY — follow-up question**: kill the previously-running scene server, or keep it and run a second stack alongside? Never kill it unasked. + - *Kill it*: kill the port-8000 dev server, have the user close the running client (never kill an Editor process yourself), then continue below. + - *Keep it*: leave it and its Explorer untouched; start a second stack on its own ports — a different dev-server port (`--port`; the launched client follows it automatically), a different MCP port (`--mcp-port`, implies `--mcp`), and `--multi-instance` so a second Explorer instance can run concurrently: + + ```bash + npm install && npm run start -- --port 8666 --multi-instance --mcp-port 8124 + ``` + + From here on use the chosen ports instead of 8000/8123 — including step 3's registration, which needs a distinct server name (e.g. `claude mcp add --transport http --scope user explorer2 http://127.0.0.1:8124/unity-explorer-mcp`; the tools then surface as `mcp__explorer2__*`). + + **No server found** — serve the scene and launch the Explorer in one command from the scene folder (keep it running in the background; if something else already holds port 8000, apply the same kill-or-keep question and port overrides as above): + + ```bash + npm install && npm run start -- --mcp + ``` + + This serves the scene at `http://127.0.0.1:8000`, auto-launches the **installed** Decentraland client connected to it with the MCP server enabled (port 8123; `--mcp-port ` picks another and implies `--mcp` — adjust the 8123 URLs in steps 1 and 3 to match), and the LSD dev server hot-reloads the running Explorer on file changes. Other useful flags: `--port ` (dev-server port; the launched client follows it automatically), `--position x,y`, `--skip-auth-screen`, `-n` (new client instance), `--multi-instance` (allow concurrent Explorer instances), `--no-client` (serve only, launch nothing). Anything after a second standalone `--` is forwarded verbatim into the launch as extra Explorer params, e.g. `npm run start -- --mcp -- --windowed-mode --resolution 1280x720` (npm consumes the first `--`). If `--mcp` is rejected as an unknown option, the scene's `@dcl/sdk-commands` predates the flag — update `@dcl/sdk`, or fall back to step 2. + + **A freshly launched Explorer needs the user to log in.** The client opens on the auth screen unless a previous session's login is still cached (`--skip-auth-screen` only skips it when a valid identity exists — a missing or expired login shows it anyway, and extra `--multi-instance` instances always ask). Tell the user to log in, then wait — step 4's polling only starts succeeding once they're through, and only then can you continue working on the scene through the MCP server. + +2. **(Alternative) Launch a specific Explorer build manually** — a local build or the Editor instead of the installed client. Serve with `npm run start -- --no-client` in step 1, then: + + ```bash + # macOS + open /path/to/Decentraland.app --args \ + --realm http://127.0.0.1:8000 --local-scene true --position 0,0 \ + --debug --skip-auth-screen --skip-version-check true \ + --mcp --windowed-mode --resolution 1280x720 + ``` + + On Windows call `Decentraland.exe` with the same arguments. Add `--disable-hud --skybox-time-enabled false --landscape-terrain-enabled false` when you want deterministic screenshots without HUD noise. + +3. **Connect the MCP server** (default port 8123): + + ```bash + claude mcp add --transport http --scope user explorer http://127.0.0.1:8123/unity-explorer-mcp + ``` + + Errors with "already exists in local config" if registered by a previous session — that's fine, nothing to do. If the current session has no `mcp__explorer__*` tools, follow "Missing tools" under Scene health & recovery below — the fix is the user reconnecting via `/mcp`, not a workaround. + +4. Wait for the world to load: poll `get_scene_state` until `loadingScreenOn` is false and the scene reports `isReady: true`. + +## The iteration loop + +Repeat until **every requirement has proof**: a screenshot or state read demonstrating it, captured from a retail camera mode (`first_person`/`third_person`, not the free camera), with `get_scene_state` healthy and no unexplained errors in the logs. + +1. **Edit** the scene TypeScript in `src/` — the LSD dev server hot-reloads the running Explorer within a few seconds. If you need a deterministic reset instead, call `reload_scene`. +2. **Confirm the scene is healthy**: `get_scene_state` — a `state` of `JavaScriptError` or `EcsError` means your code crashed the scene runtime. +3. **Read the runtime output**: `get_scene_logs` with `sinceSeq` set to the last sequence number you saw. Scene `console.log` output and exceptions land here. +4. **Look and verify**: position the view (`teleport`, `move_to`, `walk`, `look_at` — for precise framing or free-camera sweeps read [`reference/camera-and-movement.md`](reference/camera-and-movement.md)), then `screenshot` and inspect the image against what the scene code should produce. +5. **Exercise behavior**: `walk` into trigger areas, `click_entity` on interactables, `send_chat` for commands, `trigger_emote`, and re-screenshot to verify reactions. `list_scene_entities` + `get_entity_details` show the scene's ECS state when visuals aren't enough. + +**Cross-examine** every conclusion: confirm each visual claim with a state read (ECS values via `get_entity_details`, logs, `get_player_state` position), and each state claim with pixels. One channel lies routinely — colliders exist that pixels don't show, entities render invisible while their state looks healthy, animations silently don't play. The reference files call out where cross-examination is mandatory. + +**MANDATORY — camera cleanup before finishing.** NEVER leave the camera in `free` mode when you stop working (end of task, handing back to the user, or pausing for their input): always restore it with `set_camera_mode third_person` as your last camera action, and confirm via `get_player_state` → `camera.mode` if anything in between could have failed. + +## Screenshot frequency & cost + +Every screenshot returned by the MCP `screenshot` tool lands in your context as an image (~1.2k tokens at 1280×720, scaling with pixel count). Occasional captures through the tool are fine; **frequent or burst captures must go through the bundled script instead**, which saves frames to disk (zero context cost) and prints only the caption: + +```bash +scripts/screenshot.sh -o shot.jpg # single frame to a file +scripts/screenshot.sh -n 10 -i 0.5 # burst: 10 frames every 0.5s into mcp-shots/ (time-based behavior: tweens, animations) +scripts/screenshot.sh -w 640 # cheap sanity-check resolution (~4x fewer tokens when you Read it) +scripts/screenshot.sh --world-only --png # UI-less lossless frame +``` + +Paths are relative to this skill's directory; requires curl + python3; pass `-p ` when not on 8123. Then `Read` only the frames you actually need to inspect — capture many, look at few. For before/after comparisons, capture both to disk and read just those two. Use `maxWidth` 640 for quick checks and 1280 only for final verification. Captures are serialized server-side (concurrent requests are rejected), so keep burst intervals ≥ 0.2s. + +## Scene health & recovery + +- Sequence-poll logs (`sinceSeq`) instead of re-reading the whole buffer; errors survive in the buffer even if they scrolled by. +- `scene.json` changes (parcels, spawn points) are not hot-reloaded — restart the `npm run start` process, then `reload_scene`. +- After `teleport` or `reload_scene`, always re-check `get_scene_state` before interacting; readiness can lag a few seconds. +- One parcel is 16×16 m; parcel `(x, y)` spans world positions `(16x..16x+16, 16y..16y+16)`. `--position 0,0` spawns at parcel 0,0. +- If the connection drops, the client probably crashed or was closed — relaunch it the same way it was started (`npm run start -- --mcp`, or the manual launch line); the MCP endpoint URL stays the same. +- **Missing tools**: `mcp__explorer__*` tools absent in-session are recoverable (typically the Explorer wasn't running when the session started, so the registered server failed its startup connection). Ask the user to run `/mcp` and reconnect the `explorer` server — an interactive command only the user can run; a successful reconnect binds all the server's tools into the running session (verified). A plain `claude mcp add` mid-session does NOT surface tools by itself. Last resort: drive the endpoint directly with curl JSON-RPC (`POST /unity-explorer-mcp`, methods `initialize` then `tools/call`; responses may be SSE-framed, tool payloads are JSON in `result.content[0].text`, screenshots are base64 in image content blocks). +- After a hot reload the player can end up off-parcel (e.g. parcel `0,-1`); `get_scene_state` then reports a null scene and `reload_scene` fails with "no scene at the current parcel". Check `get_player_state` → `parcel`, `move_to` back inside, and the scene loads again. +- Each file save triggers a rebuild: editing usage and import in separate saves produces a transient `SceneError: X is not defined` between them. Write new modules before wiring them in, and prefer a single whole-file write for multi-part edits to one file. +- **Rapid successive saves can HARD-WEDGE the client (verified 2026-07-10).** Two saves seconds apart made the Explorer load a mid-write bundle → `SyntaxError: Invalid or unexpected token` at scene start → the scene facade is torn down and drops out of `ScenesCache`, and `get_scene_state` reports `scene: null` while standing on the parcel. From that state NOTHING recovers in-session: `reload_scene` errors ("no scene at the current parcel" — its guard and every underlying reload path need the scene still cached), `/reload` hangs until cancelled, the minimap RELOAD SCENE button just sends `/reload`, LSD file-save pushes no-op (`TryGetBySceneId` misses), and moving far off-parcel and back does not recreate the facade. Only exiting/re-entering play mode (editor) or relaunching the standalone build recovers. Prevention: batch multi-edit changes into ONE file write, and after any save landing seconds after a previous one, verify `get_scene_state` still shows a scene before saving again. +- The `teleport` tool silently no-ops in local-scene-development mode: `/goto` teleports are disallowed there (chat shows "Teleport is not allowed in local scene development mode") but the tool still answers "Arrived at (x,y)". Use `move_to` for repositioning in LSD sessions. +- The Explorer under test may be the **Unity Editor in play mode**, not a standalone Decentraland.app — check `ps aux | grep -i unity` before considering a relaunch. Never kill the editor process; recovery from a wedged client is then a user action (exit/re-enter play mode). + +## Interaction testing + +- `click_entity` presses a pointer button on a scene entity (get ids from `list_scene_entities`). The target needs a `PointerEvents` component and a collider; the aim is validated by a real camera-origin raycast, so occluders return `hit:false` + `blockedBy*` (reposition and retry) and the entity's `maxDistance` (default 10 m) applies — get close first. `upRayMissed: true` means the target moved between press and release (e.g. a door starting to swing) and the release was delivered with the press-frame hit. For GLTF entities whose collider sits away from the pivot, pass an explicit `x/y/z` aim point. The player must be standing on the scene's parcel — off-parcel clicks fail with "no running current scene". +- `walk` moves relative to the camera and requires an explicit direction: pass `directionY: 1` for forward (`directionX` strafes); omitting both errors with "directionX and directionY must not both be zero". +- Collider checks beat pixels for physics (cross-examine): `look_at` straight at the target, `walk` forward, then compare `get_player_state` positions to prove passage or blockage. +- Trigger areas fire `onTriggerEnter` immediately after `reload_scene` if the player is already standing inside one — reposition the player outside all triggers before testing enter/exit sequencing (and treat post-reload trigger logs as stale state, not gameplay). + +## Improving this skill + +Two rules: + +**1. Learned something the skill didn't tell you?** A flag that turned out to be required, a timing quirk, a better verification pattern, or information here that proved wrong — edit this skill in place before finishing your session. Keep additions terse and verified (facts you observed, not speculation), and file them where their branch lives: setup, health/recovery, and interaction facts in this SKILL.md; camera and navigation facts in `reference/camera-and-movement.md`; model and import facts in `reference/assets.md`; rendering and visual-tuning facts in `reference/visuals.md`. The canonical copy lives in the unity-explorer repo at `.claude/skills/mcp-scene-iteration/`; if you are running from a copy (e.g. `~/.claude/skills/`), apply the same edit to the canonical copy too when the repo is accessible, or tell the user to sync it. + +**2. Missing a capability?** If the loop is blocked because no existing MCP tool can do what you need (e.g. pressing a specific key, reading a value no tool exposes), do NOT work around it by modifying the Explorer client, the MCP server, or the unity-explorer repo yourself. Stop and prompt the user with a concrete tool proposal: + +- proposed tool name and one-line purpose +- input arguments (names, types, defaults) +- expected output shape +- the blocked use case, and why the existing tools can't cover it + +The user decides whether and when to implement it. **MANDATORY: implementing an approved tool must go through plan mode first** — whichever session does the implementation starts in plan mode, researches the unity-explorer codebase (the server lives under `Explorer/Assets/DCL/McpServer/` — see `docs/mcp-automation.md` → Implementation map), and presents the plan for user approval before writing any code. Also append the proposal to the "Wanted tools" list below so it isn't lost if the user defers. + +## Wanted tools + +Proposals from agent sessions — name, purpose, blocked use case. Remove entries once implemented. + +- **recover_scene** — force-recreate the scene at the player's parcel when it has dropped out of `ScenesCache` (`get_scene_state` → `scene: null`; the hard-wedge state described above, where every existing reload path needs the cached facade and the session is dead until the user restarts play mode). Inputs: `timeoutSec?: number` (default 30). Output: same shape as `reload_scene`. Implementation lead: clear failed `AssetPromise` state on the definition entity and reset `StaticScenePointers.Promise` on the realm entity so the static-pointer systems re-resolve — the same reset `ECSReloadScene.DisposeAndRestartAsync` already performs for LSD, minus the requirement that a live scene exists. diff --git a/.claude/skills/mcp-scene-iteration/reference/assets.md b/.claude/skills/mcp-scene-iteration/reference/assets.md new file mode 100644 index 00000000000..c0a4cd8183a --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/assets.md @@ -0,0 +1,22 @@ +# 3D asset reference + +Read before placing, downloading, converting, or exporting any 3D model. + +## Picking from the sdk-skills catalog + +- Fetch the `preview:` thumbnail BEFORE downloading — some models render near-black or broken even in their own previews (e.g. arcade-cabinet-atari). Use the exact `[anim: ...]` clip names; a wrong clip name fails silently (no error, no motion) — cross-examine: burst-capture (`-n 3 -i 1`) and diff frames to prove an animation is actually running. +- Catalog dims/pivots lie: bounding boxes can include baked animation paths or outline meshes, pivots can sit mid-crown (palms) or nowhere near the mesh (a baked drive path carried one car's mesh entirely off-parcel — swap such models rather than debug them). Place → screenshot → adjust is faster than trusting listed sizes. + +## Downloading models + +- Free CC0 model sources that download cleanly via curl: kenney.nl (zip URL is on the asset page, FBX+OBJ+GLTF inside), and itch.io free packs via the scripted flow: POST `/download_url` with the page's csrf_token → GET the returned key URL → grab `data-upload_id` → POST `/file/?source=game_download` → signed CDN URL (expires ~60s, download immediately). +- Downloaded GLBs into the scene folder hot-load without restarting the dev server. Many props ship with no colliders — cross-examine solidity: walk onto them and check the player's `y` via `get_player_state`; add `visibleMeshesCollisionMask: 3` for anything that should be solid. + +## Blender authoring & conversion + +- Blender-authored GLBs work end-to-end: export with `bpy.ops.export_scene.gltf(use_selection=True)` straight into the scene's Models folder (hot-loads like any file change). Set the object origin to bottom-center before export (`cursor to (0,0,0)` + `origin_set(type='ORIGIN_CURSOR')` with geometry built up from z=0) so `position.y = 0` grounds the model, and `transform_apply` rotation/scale for clean transforms. Principled BSDF emissive (Emission Color + Strength) renders as expected in Explorer, including the zero-channel neon saturation rule (see `visuals.md`). +- Converting downloaded FBX/OBJ to GLB in Blender works, with three verified traps: (1) FBX materials can import with Principled Alpha = 0 (FBX transparency-factor quirk, seen on Quaternius packs) — the GLB then has `alphaMode: MASK` with baseColor alpha 0 and the model is INVISIBLE in Explorer while its entity, tween and logs all look healthy; force Alpha=1 + `blend_method='OPAQUE'` before export, and when a GLB renders nothing, parse its JSON chunk (nodes/materials) instead of guessing. (2) The glTF exporter's default animation mode exports EVERY action in the .blend that fits the armature, so clips from other imported models leak into each GLB; use `export_animation_mode='ACTIVE_ACTIONS'` with the right action active (exports one clip named `Animation`). (3) Some kits (e.g. Kenney furniture) ship ASCII FBX which Blender refuses — run `file *.fbx` first and fall back to the kit's OBJ folder. Skinned meshes respect entity Transform scale, so oversize rigs can be scaled at the entity. + +## Composite gotchas + +- Composite-authored box primitives need `"box": {"uvs": []}` in `core::MeshRenderer` — a bare `"box": {}` crashes `sdk-commands build` with `TypeError: message.uvs is not iterable`. diff --git a/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md b/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md new file mode 100644 index 00000000000..0d14eb4b36f --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/camera-and-movement.md @@ -0,0 +1,18 @@ +# Camera & movement reference + +Read before framing screenshots, navigating precise lines, or inspecting a build from multiple viewpoints. + +## Aiming the camera + +- `move_to`'s `lookAt*` params orient the avatar but NOT the third-person camera; `screenshot` and `walk` follow the camera. Call the standalone `look_at` tool (it aligns camera yaw — confirm via `get_player_state` → `camera.rotationEuler.y`) before walking a precise line or framing a screenshot. +- `look_at` lines the third-person camera up through the avatar, so the avatar occludes exactly the thing you framed. To photograph a subject, first `move_to` a spot offset sideways from the camera→subject line, then `look_at`. + +## Free camera + +- The free camera is the fastest way to inspect a scene from many points of view: `set_camera_pose` places it at any absolute position, optionally aims it (`lookAt*`) and sets `fov`, auto-entering free mode. Repositioning while already free is instant (~200ms), so sweep a build cheaply — aerial plan view, each facade, eye-level details, interiors — capturing to disk between calls, instead of walking the player around. `look_at` also works in free mode (aims from the camera's own position), and the free camera stays put while the player moves, so you can even watch the avatar walk through the scene from a fixed vantage. Entering free from another mode blends over ~2-3s (the tool waits and reports `settled`). +- The free camera is a debug view, not what players see. To confirm the end-user experience, switch back to the real modes — `set_camera_mode` `first_person` / `third_person` / `drone_view` are exactly the cameras retail users have — and re-check framing, avatar occlusion, and interaction reach from there (e.g. verify a hover target is actually visible and clickable at player height, not just from a flattering freecam angle). **MANDATORY: never leave the camera in free mode when you finish working** — always restore `set_camera_mode third_person` as your last camera action. `set_camera_mode` respects scene locks and errors truthfully — check `get_player_state` → `camera.modeChangeAllowed` first; `false` inside a `CameraModeArea`/scene virtual camera is correct behavior worth verifying, not a tool failure. `screenshot` works in any mode. + +## Precise navigation + +- Collider-bump navigation moves the player along precise lines reliably: `look_at` a point past where you want to end up, `walk` with generous `seconds`, and let a blocking collider stop the player — the returned `endPosition` lands ~0.38m (capsule radius) short of the collider face, a deterministic waypoint for the next leg with no duration tuning. Timing legs precisely is fragile (jog ramp-up varies effective speed); overshooting into a collider is not. Only a final leg through a gap with nothing beyond it needs a tight duration, or the player runs off-parcel. +- Measured locomotion speeds (flat ground): jog ≈6.5 m/s once ramped, `kind: "walk"` ≈1.4 m/s. To stop at a precise point no collider will stop you at, take one timed jog leg, read `endPosition`, then micro-correct with 0.5-1.2s `kind: "walk"` legs — position feedback plus slow corrections converges in 1-2 iterations where a single timed jog leg won't. diff --git a/.claude/skills/mcp-scene-iteration/reference/visuals.md b/.claude/skills/mcp-scene-iteration/reference/visuals.md new file mode 100644 index 00000000000..8325bac0895 --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/reference/visuals.md @@ -0,0 +1,9 @@ +# Visual tuning reference + +Read before tuning emissives/bloom, UI overlays, skybox time, or judging visibility of thin geometry. + +- `screenshot`'s `worldOnly` (and the script's `--world-only`) renders with full post-processing, bloom included (fixed + verified 2026-07-06: the capture target was LDR, which clamped emissives and starved bloom — it is HDR now). Emissive/glow looks are judgeable from world-only frames, with one caveat: halo spread reads slightly tighter than the full-view capture (render-resolution difference), so do final pixel-exact glow tuning on full-view frames. +- Neon emissive recipe (verified with bloom, sunset skybox): pin one emissive channel to exactly 0 (e.g. `(1, 0, 0.6)` magenta) — a small minor channel (0.15-0.2) times `emissiveIntensity` 6+ whites the whole surface out. Then weight by luminance and background: green-heavy hues (cyan) bloom to white far faster than magenta and additively desaturate against the pink haze (cyan + pink = white), so dim or blue-shift them (`(0, 0.35, 0.8)` works); magenta reinforces the sunset and stays saturated at intensity 6-15. A large on-screen emitter always whites out at its core — the hue lives in the halo, which is what a real neon tube looks like; judge hue from gameplay distance, not close-ups. Bloom already provides the halo, so an emitter needs no alpha-blended "glow shell" box around it — a shell just reads as a grey display case. +- UiBackground texture entities render far stronger than a near-zero `color` alpha suggests (a 0.05-alpha noise overlay still fully obscured the image beneath). For flicker/glitch overlays, mount/unmount the entity conditionally (`{cond ? : null}`) instead of animating alpha near zero. +- Thin geometry (a door panel ~0.1m) is invisible edge-on: a panel "open" at ~90-110° reads as a sliver from the front. Cross-examine pose via ECS rotation (`get_entity_details` quaternion) as well as pixels, and prefer open angles like ~135° when front visibility matters. +- `SkyboxTime.create(engine.RootEntity, { fixedTime })` pins the scene to a permanent time of day regardless of launch flags. diff --git a/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh b/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh new file mode 100755 index 00000000000..1b4ed6a879b --- /dev/null +++ b/.claude/skills/mcp-scene-iteration/scripts/screenshot.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# Capture Explorer screenshots to disk via the embedded MCP server, without spending agent context. +# Frames are saved as files; only the caption (resolution + parcel) is printed. Read a frame file +# only when you actually need to inspect it. +# +# Usage: screenshot.sh [options] +# -o, --out FILE output file (single capture; extension follows quality) +# -d, --dir DIR output directory (default: mcp-shots/; used for bursts and when -o is omitted) +# -n, --count N number of frames to capture (default: 1) +# -i, --interval SEC seconds between frames in a burst (default: 0.5; keep >= 0.2, captures are serialized) +# -w, --max-width PX maxWidth passed to the tool (default: 1280; use 640 for cheap sanity checks) +# --png capture PNG instead of JPG +# --world-only exclude UI overlays (worldOnly: true) +# -p, --port PORT MCP server port (default: 8123) +# +# Requires: curl, python3. The Explorer must be running with --mcp. + +set -euo pipefail + +PORT=8123 +MAX_WIDTH=1280 +QUALITY=jpg +WORLD_ONLY=false +COUNT=1 +INTERVAL=0.5 +OUT="" +OUT_DIR="mcp-shots" + +while [[ $# -gt 0 ]]; do + case "$1" in + -o|--out) OUT="$2"; shift 2 ;; + -d|--dir) OUT_DIR="$2"; shift 2 ;; + -n|--count) COUNT="$2"; shift 2 ;; + -i|--interval) INTERVAL="$2"; shift 2 ;; + -w|--max-width) MAX_WIDTH="$2"; shift 2 ;; + --png) QUALITY=png; shift ;; + --world-only) WORLD_ONLY=true; shift ;; + -p|--port) PORT="$2"; shift 2 ;; + -h|--help) sed -n '2,17p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "unknown argument: $1 (see --help)" >&2; exit 2 ;; + esac +done + +capture_one() { + local target_file="$1" + + local payload + payload=$(printf '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"screenshot","arguments":{"maxWidth":%s,"quality":"%s","worldOnly":%s}}}' \ + "$MAX_WIDTH" "$QUALITY" "$WORLD_ONLY") + + # NOTE: response must land in a file, not a pipe into `python3 - </dev/null || true + capture_one "$OUT" + exit 0 +fi + +mkdir -p "$OUT_DIR" +stamp=$(date +%Y%m%d-%H%M%S) + +for ((frame = 1; frame <= COUNT; frame++)); do + capture_one "${OUT_DIR}/shot-${stamp}-$(printf '%03d' "$frame").${QUALITY}" + + if (( frame < COUNT )); then + sleep "$INTERVAL" + fi +done diff --git a/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs b/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs index 26aaa886bb3..8034a14805d 100644 --- a/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs +++ b/Explorer/Assets/DCL/Character/CharacterCamera/CinemachineExtensions.cs @@ -1,4 +1,5 @@ -using DCL.CharacterCamera.Components; +using Cinemachine; +using DCL.CharacterCamera.Components; using UnityEngine; namespace DCL.CharacterCamera @@ -16,9 +17,10 @@ public static void ForceFirstPersonCameraLookAt(this ICinemachinePreset cinemach { if (cinemachinePreset.FirstPersonCameraData.POV == null) return; - (float horizontalAxis, float verticalAxis) = GetHorizontalAndVerticalAxisForIntent(lookAtIntent); - cinemachinePreset.FirstPersonCameraData.POV.m_HorizontalAxis.Value = horizontalAxis; - cinemachinePreset.FirstPersonCameraData.POV.m_VerticalAxis.Value = verticalAxis; + // POV axes are in degrees, unlike the 0..1 orbit value the FreeLook rigs consume. + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, lookAtIntent.PlayerPosition); + cinemachinePreset.FirstPersonCameraData.POV.m_HorizontalAxis.Value = yawDegrees; + cinemachinePreset.FirstPersonCameraData.POV.m_VerticalAxis.Value = pitchDegrees; } public static void ForceDroneCameraLookAt(this ICinemachinePreset cinemachinePreset, CameraLookAtIntent lookAtIntent) @@ -28,23 +30,58 @@ public static void ForceDroneCameraLookAt(this ICinemachinePreset cinemachinePre cinemachinePreset.DroneViewCameraData.Camera.m_YAxis.Value = verticalAxis; } - private static (float, float) GetHorizontalAndVerticalAxisForIntent(CameraLookAtIntent lookAtIntent) + /// + /// Places the free camera at an absolute world position (and optionally sets its field of view). + /// The free camera's position is its vcam transform, the same mechanism free-fly input moves it through. + /// + public static void ForceFreeCameraPose(this ICinemachinePreset cinemachinePreset, Vector3 position, float? fov = null) { - var eulerDir = Vector3.zero; - var cameraTarget = lookAtIntent.LookAtTarget; - float horizontalAxisLookAt = lookAtIntent.PlayerPosition.y - cameraTarget.y; - var verticalAxisLookAt = new Vector3(cameraTarget.x - lookAtIntent.PlayerPosition.x, 0, cameraTarget.z - lookAtIntent.PlayerPosition.z); + cinemachinePreset.FreeCameraData.Camera.transform.position = position; - if (verticalAxisLookAt is { x: 0, y: 0, z: 0 }) - verticalAxisLookAt = Vector3.forward; + if (fov.HasValue) + cinemachinePreset.FreeCameraData.Camera.m_Lens.FieldOfView = fov.Value; + } + + public static void ForceFreeCameraLookAt(this ICinemachinePreset cinemachinePreset, CameraLookAtIntent lookAtIntent) + { + CinemachinePOV? pov = cinemachinePreset.FreeCameraData.POV; - eulerDir.y = Vector3.SignedAngle(Vector3.forward, verticalAxisLookAt, Vector3.up); - eulerDir.x = Mathf.Atan2(horizontalAxisLookAt, verticalAxisLookAt.magnitude) * Mathf.Rad2Deg; + if (pov == null) + return; + + // The free camera is detached from the player, so the aim originates at the camera itself. + Vector3 origin = cinemachinePreset.FreeCameraData.Camera.transform.position; + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, origin); + + pov.m_HorizontalAxis.Value = yawDegrees; + pov.m_VerticalAxis.Value = pitchDegrees; + } + + private static (float, float) GetHorizontalAndVerticalAxisForIntent(CameraLookAtIntent lookAtIntent) + { + (float yawDegrees, float pitchDegrees) = GetYawPitchDegrees(lookAtIntent.LookAtTarget, lookAtIntent.PlayerPosition); //value range 0 to 1, being 0 the bottom orbit and 1 the top orbit - float yValue = Mathf.InverseLerp(-90, 90, eulerDir.x); + float yValue = Mathf.InverseLerp(-90, 90, pitchDegrees); + + return (yawDegrees, yValue); + } + + /// + /// Yaw/pitch in degrees (Unity euler convention: positive pitch looks down) that point from origin at target. + /// + private static (float yawDegrees, float pitchDegrees) GetYawPitchDegrees(Vector3 target, Vector3 origin) + { + float heightDelta = origin.y - target.y; + var flatDirection = new Vector3(target.x - origin.x, 0, target.z - origin.z); + + if (flatDirection is { x: 0, y: 0, z: 0 }) + flatDirection = Vector3.forward; + + float yawDegrees = Vector3.SignedAngle(Vector3.forward, flatDirection, Vector3.up); + float pitchDegrees = Mathf.Atan2(heightDelta, flatDirection.magnitude) * Mathf.Rad2Deg; - return (eulerDir.y, yValue); + return (yawDegrees, pitchDegrees); } } } diff --git a/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs b/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs index a89d0102fb8..57bd1e99ac7 100644 --- a/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs +++ b/Explorer/Assets/DCL/Character/CharacterCamera/Systems/ApplyCinemachineCameraInputSystem.cs @@ -81,6 +81,8 @@ private void ForceLookAt(in Entity entity, in CameraComponent camera, ref ICinem cinemachinePreset.ForceFirstPersonCameraLookAt(lookAtIntent); break; case CameraMode.Free: + cinemachinePreset.ForceFreeCameraLookAt(lookAtIntent); + break; case CameraMode.ThirdPerson: cinemachinePreset.ForceThirdPersonCameraLookAt(lookAtIntent); break; diff --git a/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs b/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs index ec0c6946612..3088c98eccc 100644 --- a/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs +++ b/Explorer/Assets/DCL/FeatureFlags/FeaturesRegistry.cs @@ -71,6 +71,7 @@ public FeaturesRegistry( [FeatureId.AB_DEPS_DIGEST_CACHE_KEY] = featureFlags.IsEnabled(FeatureFlagsStrings.AB_DEPS_DIGEST_CACHE_KEY), [FeatureId.BYTE_WEIGHTED_LOADING_PROGRESS] = appArgs.ResolveFeatureFlagArg(AppArgsFlags.BYTE_WEIGHTED_LOADING_PROGRESS, featureFlags.IsEnabled(FeatureFlagsStrings.BYTE_WEIGHTED_LOADING_PROGRESS) || isEditor), [FeatureId.HARDWARE_FINGERPRINT] = appArgs.ResolveFeatureFlagArg(AppArgsFlags.HARDWARE_FINGERPRINT, featureFlags.IsEnabled(FeatureFlagsStrings.HARDWARE_FINGERPRINT)), + [FeatureId.MCP_SERVER] = appArgs.HasFlag(AppArgsFlags.MCP) || appArgs.HasFlag(AppArgsFlags.MCP_PORT), // Note: COMMUNITIES feature is not cached here because it depends on user identity }); @@ -207,5 +208,6 @@ public enum FeatureId HARDWARE_FINGERPRINT = 66, USER_CREDITS = 67, CREDITS_WEARABLE_PURCHASE = 68, + MCP_SERVER = 69, } } diff --git a/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs b/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs index 3fadae5145b..597f5650c29 100644 --- a/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs +++ b/Explorer/Assets/DCL/Infrastructure/Global/AppArgs/AppArgsFlags.cs @@ -106,6 +106,16 @@ public static class AppArgsFlags public const string MULTIPLE_RUNNING_INSTANCES = "multi-instance"; public const string ALTTESTER = "alttester"; + /// + /// Starts the embedded MCP (Model Context Protocol) server on 127.0.0.1 so coding agents can drive the client. + /// + public const string MCP = "mcp"; + + /// + /// Overrides the port the embedded MCP server listens on (implies ). + /// + public const string MCP_PORT = "mcp-port"; + public const string REPORT_USER = "report-user"; public const string AVATAR_CONTEXT_MENU = "avatar-context-menu"; diff --git a/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs b/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs index 08c2cbc3238..e20941515a7 100644 --- a/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs +++ b/Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs @@ -1,4 +1,5 @@ using Arch.Core; +using CrdtEcsBridge.RestrictedActions; using Cysharp.Threading.Tasks; using DCL.ApplicationGuards; using DCL.AssetsProvision; @@ -24,6 +25,7 @@ using DCL.LOD.Systems; using DCL.MarketplaceCredits; using DCL.MarketplaceCredits.Purchase; +using DCL.McpServer.Systems; using DCL.Multiplayer.Connections.Messaging.Hubs; using DCL.Multiplayer.Connections.RoomHubs; using DCL.Multiplayer.Emotes; @@ -85,8 +87,6 @@ public class DynamicWorldContainer : DCLWorldContainer public IGlobalRealmController RealmController { get; } - public IRealmNavigator RealmNavigator { get; } - public GlobalWorldFactory GlobalWorldFactory { get; } public IReadOnlyList GlobalPlugins { get; } @@ -112,7 +112,6 @@ public class DynamicWorldContainer : DCLWorldContainer private DynamicWorldContainer( UIShellContainer uiShellContainer, IGlobalRealmController realmController, - IRealmNavigator realmNavigator, GlobalWorldFactory globalWorldFactory, IReadOnlyList globalPlugins, IReadOnlyList worldPlugins, @@ -130,7 +129,6 @@ private DynamicWorldContainer( { this.uiShellContainer = uiShellContainer; RealmController = realmController; - RealmNavigator = realmNavigator; GlobalWorldFactory = globalWorldFactory; GlobalPlugins = globalPlugins; WorldPlugins = worldPlugins; @@ -842,6 +840,23 @@ await MapRendererContainer globalPlugins.Add(lodContainer.RoadPlugin); } + if (FeaturesRegistry.Instance.IsEnabled(FeatureId.MCP_SERVER)) + globalPlugins.Add(new McpServerPlugin( + appArgs, + new GlobalWorldActions(globalWorld, playerEntity, localSceneDevelopment, bootstrapContainer.UseRemoteAssetBundles, FeaturesRegistry.Instance.IsEnabled(FeatureId.SELF_PREVIEW_BUILDER_COLLECTIONS)), + chatContainer.ChatMessagesBus, + staticContainer.ScenesCache, + commsContainer.CurrentSceneInfo, + staticContainer.LoadingStatus, + realmNavigatorContainer.WorldInfoHub, + realmContainer.ReloadSceneController, + bootstrapContainer.DiagnosticsContainer, + exposedGlobalDataContainer.ExposedCameraData, + staticContainer.EntityCollidersGlobalCache, + coroutineRunner, + globalWorld, + localSceneDevelopment)); + if (FeaturesRegistry.Instance.IsEnabled(FeatureId.LOCAL_SCENE_DEVELOPMENT) || FeaturesRegistry.Instance.IsEnabled(FeatureId.SELF_PREVIEW_BUILDER_COLLECTIONS)) globalPlugins.Add(new GlobalGLTFLoadingPlugin(staticContainer.WebRequestsContainer.WebRequestController, staticContainer.RealmData, wearableContainer.BuilderContentURL.Value, localSceneDevelopment, staticContainer.ComponentsContainer.ComponentPoolsRegistry.RootContainerTransform())); @@ -1009,7 +1024,6 @@ await MapRendererContainer var container = new DynamicWorldContainer( uiShellContainer, realmContainer.RealmController, - realmNavigator, globalWorldFactory, globalPlugins, worldPlugins, diff --git a/Explorer/Assets/DCL/Infrastructure/Utility/DecentralandUrls/IDecentralandUrlsSource.cs b/Explorer/Assets/DCL/Infrastructure/Utility/DecentralandUrls/IDecentralandUrlsSource.cs index 7c0a052a375..5fa3dbed8ea 100644 --- a/Explorer/Assets/DCL/Infrastructure/Utility/DecentralandUrls/IDecentralandUrlsSource.cs +++ b/Explorer/Assets/DCL/Infrastructure/Utility/DecentralandUrls/IDecentralandUrlsSource.cs @@ -6,6 +6,13 @@ public interface IDecentralandUrlsSource const string LAUNCHER_DOWNLOAD_URL = "https://explorer-artifacts.decentraland.org/launcher-rust"; const string LEGACY_LAUNCHER_DOWNLOAD_URL = "https://explorer-artifacts.decentraland.org/launcher/dcl"; + /// + /// Local MCP server endpoint template; {0} is the port (see "--mcp-port"). Docs and scripts restate the + /// path (they cannot reference this const): docs/mcp-automation.md, docs/app-arguments.md, + /// .claude/skills/mcp-scene-iteration/ — keep in sync. + /// + const string LOCAL_MCP_ENDPOINT_URL = "http://127.0.0.1:{0}/unity-explorer-mcp"; + /// /// Get a raw url without caching at any moment (without dependency on FF) /// diff --git a/Explorer/Assets/DCL/McpServer.meta b/Explorer/Assets/DCL/McpServer.meta new file mode 100644 index 00000000000..f4a20233b14 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 99595f3e5da16754fa6419d428a933e8 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Components.meta b/Explorer/Assets/DCL/McpServer/Components.meta new file mode 100644 index 00000000000..3168cc8ff37 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: edabe99ee7c5e413e92d095e4480a477 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs new file mode 100644 index 00000000000..3bd48b02b3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs @@ -0,0 +1,36 @@ +using Cysharp.Threading.Tasks; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Core; +using UnityEngine; + +namespace DCL.McpServer.Components +{ + /// + /// Held movement input requested by the MCP walk tool. While present on the player entity, + /// re-asserts it into every frame. + /// + public struct McpEcsMovementOverride : IMcpEcsRequest + { + /// + /// Normalized camera-relative axes (x = strafe, y = forward). + /// + public Vector2 Axes; + + public MovementKind Kind; + + /// + /// Value of Time.time at which the hold expires. + /// + public float EndTime; + + /// + /// Requests a single jump; consumed on the first frame of the hold. + /// + public bool JumpRequested; + + /// + /// Completed by the system when the hold expires or is preempted by a newer request. + /// + public UniTaskCompletionSource? Completion { get; set; } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta new file mode 100644 index 00000000000..0f86dbd6f92 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpMovementOverride.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 64ba45e1952cb4631b46b2d0dd9688e2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Components/McpPointerEventIntent.cs b/Explorer/Assets/DCL/McpServer/Components/McpPointerEventIntent.cs new file mode 100644 index 00000000000..915f769ba86 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpPointerEventIntent.cs @@ -0,0 +1,86 @@ +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Core; +using UnityEngine; +using RaycastHit = UnityEngine.RaycastHit; + +namespace DCL.McpServer.Components +{ + /// + /// Present on the player entity while a single agent-requested pointer event awaits delivery. + /// McpPointerEventSystem validates the aim with a physics raycast, appends the event to the target's + /// and removes the component. The request + /// is immutable; a full click is composed by the click_entity tool from two intents — a press, then a + /// release carrying the press handoff. A request the simulation never picks up is + /// removed by the tool-side timeout. + /// + public struct McpEcsPointerEventIntent : IMcpEcsRequest + { + /// Arch entity id in the current scene world; -1 when aiming at an explicit world point. + public readonly int TargetEntityId; + + /// Explicit world-space aim point; when null the aim is the target's collider center. + public readonly Vector3? AimPoint; + + public readonly InputAction Button; + + /// PetDown or PetUp. + public readonly PointerEventType EventType; + + /// + /// Set on the release leg of a click: the press this release must stay ordered after. Delivery waits + /// until the scene has advanced past the press tick, is bound to the world that received the press, + /// and falls back to the press-frame hit when the fresh ray no longer reaches the target. + /// + public readonly McpPressHandoff? Press; + + public UniTaskCompletionSource? Completion { get; set; } + + public McpEcsPointerEventIntent(int targetEntityId, Vector3? aimPoint, InputAction button, PointerEventType eventType, McpPressHandoff? press = null) + { + TargetEntityId = targetEntityId; + AimPoint = aimPoint; + Button = button; + EventType = eventType; + Press = press; + Completion = null; + } + } + + /// + /// Where a delivered pointer event landed. Handed back inside and + /// passed verbatim on the release intent of a click. In-process only, never serialized. + /// + public struct McpPressHandoff + { + public Arch.Core.World World; + public Arch.Core.Entity Entity; + public uint Tick; + public RaycastHit Hit; + public Ray Ray; + } + + /// Outcome of a synthetic pointer event or click, serialized by the click_entity tool. + public struct McpPointerClickResult + { + public bool Hit; + public string? FailureReason; + public int SceneEntityId; + public int CrdtEntityId; + public string? HoverText; + public Vector3 HitPoint; + public float Distance; + public int? BlockedByEntityId; + public int? BlockedByCrdtId; + public string? BlockedByColliderName; + + /// + /// The release ray no longer hit the target (it moved after the press): PetUp was delivered with the + /// press-frame hit, or not at all when is also false. + /// + public bool UpRayMissed; + + /// Where the event landed; the click_entity tool passes it back on the release leg of a click. In-process only. + public McpPressHandoff? Press; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Components/McpPointerEventIntent.cs.meta b/Explorer/Assets/DCL/McpServer/Components/McpPointerEventIntent.cs.meta new file mode 100644 index 00000000000..8e6d348bae3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Components/McpPointerEventIntent.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 203dec8323e545c8a510e5883a59a4f6 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core.meta b/Explorer/Assets/DCL/McpServer/Core.meta new file mode 100644 index 00000000000..df86f87de1c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: ea5698d1ae0de4f699345c74dc6f99b6 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs new file mode 100644 index 00000000000..bb7046d7182 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs @@ -0,0 +1,258 @@ +using Cysharp.Threading.Tasks; +using DCL.Diagnostics; +using DCL.Multiplayer.Connections.DecentralandUrls; +using System; +using System.IO; +using System.Net; +using System.Text; +using System.Threading; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Core +{ + /// + /// Minimal MCP Streamable HTTP transport on top of , bound to 127.0.0.1 only. + /// POST carries a single JSON-RPC message; GET (server-initiated stream) is not supported and returns 405. + /// + public class McpHttpServer : IDisposable + { + private const int MAX_BODY_BYTES = 1024 * 1024; + + private readonly McpJsonRpcDispatcher dispatcher; + private readonly int port; + + // The session maps to the Explorer process, not to a client: this server is stateless and drives one + // shared world, so every request operates on the same state and there is nothing session-scoped to isolate. + // Multiple agents on one server intentionally share this id because they drive the same world; a separate + // session means a separate Explorer instance (separate port, separate id). Hence one id for the process + // lifetime, echoed on every response, and no need to validate the incoming Mcp-Session-Id. + private readonly string sessionId = Guid.NewGuid().ToString("N"); + + private HttpListener? listener; + + /// The localhost URL the server listens on, e.g. http://127.0.0.1:8123/unity-explorer-mcp. + public string EndpointUrl { get; } + + public McpHttpServer(McpToolsRegistry toolsRegistry, int port) + { + this.dispatcher = new McpJsonRpcDispatcher(toolsRegistry, Application.version); + this.port = port; + EndpointUrl = string.Format(IDecentralandUrlsSource.LOCAL_MCP_ENDPOINT_URL, port); + } + + public void Dispose() + { + if (listener == null) return; + + try + { + listener.Stop(); + listener.Close(); + } + catch (ObjectDisposedException) { } + + listener = null; + } + + public bool TryStart() + { + var newListener = new HttpListener(); + newListener.Prefixes.Add($"{EndpointUrl}/"); // HttpListener requires the trailing slash + + try { newListener.Start(); } + catch (Exception e) when (e is HttpListenerException or InvalidOperationException) + { + ReportHub.LogError(ReportCategory.MCP, $"Cannot start the MCP server on port {port} (already in use? pass a different --mcp-port): {e.Message}"); + newListener.Close(); + return false; + } + + listener = newListener; + ReportHub.Log(LogType.Log, ReportCategory.MCP, $"MCP server listening on {EndpointUrl}"); + return true; + } + + public async UniTaskVoid RunAsync(CancellationToken ct) + { + await DCLTask.SwitchToThreadPool(); + + // Capture once: a concurrent Dispose() on the main thread nulls the field, and re-reading it between + // the guard and GetContextAsync would throw an unfiltered NRE. Dispose still stops/closes this same + // instance, so a parked GetContextAsync surfaces as one of the caught exceptions below. + HttpListener? local = listener; + + while (!ct.IsCancellationRequested && local is { IsListening: true }) + { + HttpListenerContext context; + + try { context = await local.GetContextAsync(); } + catch (Exception e) when (e is HttpListenerException or ObjectDisposedException or InvalidOperationException) + { + // The listener was stopped or disposed; end the accept loop. + break; + } + + HandleRequestAsync(context, ct).Forget(); + } + } + + private async UniTaskVoid HandleRequestAsync(HttpListenerContext context, CancellationToken ct) + { + // The accept loop runs this handler inline until the first suspension; hop off it before the + // synchronous body read so one slow client cannot stall accepting new connections. + await DCLTask.SwitchToThreadPool(); + + try + { + if (!IsAllowed(context.Request.Headers["Origin"])) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.Forbidden, sessionId); + return; + } + + switch (context.Request.HttpMethod) + { + case "POST": + await HandlePostAsync(context, ct); + break; + case "DELETE": + // Session termination is accepted but stateless: nothing to clean up. + context.Response.WriteEmptyAndClose(HttpStatusCode.OK, sessionId); + break; + default: + context.Response.WriteEmptyAndClose(HttpStatusCode.MethodNotAllowed, sessionId); + break; + } + } + catch (OperationCanceledException) + { + TryAbort(context); + } + catch (Exception e) + { + ReportHub.LogException(e, ReportCategory.MCP); + TryWriteInternalError(context); + } + } + + private async UniTask HandlePostAsync(HttpListenerContext context, CancellationToken ct) + { + if (context.Request.ContentLength64 > MAX_BODY_BYTES) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.RequestEntityTooLarge, sessionId); + return; + } + + // The Content-Length check above is only a fast reject; a chunked request reports ContentLength64 == -1 + // and bypasses it, so the read itself stays capped to keep the body bounded regardless. + if (!TryReadBodyWithinCap(context.Request, out string requestJson)) + { + context.Response.WriteEmptyAndClose(HttpStatusCode.RequestEntityTooLarge, sessionId); + return; + } + + string? responseJson = await dispatcher.DispatchAsync(requestJson, ct); + + if (responseJson == null) + { + // Notifications get 202 Accepted with no body. + context.Response.WriteEmptyAndClose(HttpStatusCode.Accepted, sessionId); + return; + } + + byte[] payload = Encoding.UTF8.GetBytes(responseJson); + + context.Response.WithMcpHeaders(sessionId); + context.Response.StatusCode = (int)HttpStatusCode.OK; + context.Response.ContentType = "application/json; charset=utf-8"; + context.Response.ContentLength64 = payload.Length; + await context.Response.OutputStream.WriteAsync(payload, 0, payload.Length, CancellationToken.None); + context.Response.Close(); + } + + /// + /// Reads the request body synchronously (blocking the current thread-pool thread) through a stack + /// buffer, so heap allocations stay proportional to the actual body size instead of the cap. + /// Returns false when the body exceeds ; is + /// then empty and the request must be rejected. + /// + private static bool TryReadBodyWithinCap(HttpListenerRequest request, out string body) + { + using Stream input = request.InputStream; + + // ContentLength64 is -1 for chunked requests, so it can size the accumulator only when declared. + var accumulated = new MemoryStream(request.ContentLength64 > 0 ? (int)request.ContentLength64 : 4 * 1024); + Span chunk = stackalloc byte[16 * 1024]; + + int bytesRead; + + while ((bytesRead = input.Read(chunk)) > 0) + { + if (accumulated.Length + bytesRead > MAX_BODY_BYTES) + { + body = string.Empty; + return false; + } + + accumulated.Write(chunk[..bytesRead]); + } + + body = Encoding.UTF8.GetString(accumulated.GetBuffer(), 0, (int)accumulated.Length); + return true; + } + + private void TryWriteInternalError(HttpListenerContext context) + { + try + { + context.Response.WriteEmptyAndClose(HttpStatusCode.InternalServerError, sessionId); + } + catch (Exception) + { + // The response may already be closed or the client gone; nothing else to do. + } + } + + private static void TryAbort(HttpListenerContext context) + { + try { context.Response.Abort(); } + catch (Exception) + { + // Ignored: aborting a torn-down connection during shutdown. + } + } + + private static bool IsAllowed(string? origin) + { + if (string.IsNullOrEmpty(origin)) + return true; + + if (!Uri.TryCreate(origin, UriKind.Absolute, out Uri? originUri)) + return false; + + if (originUri.Scheme != Uri.UriSchemeHttp && originUri.Scheme != Uri.UriSchemeHttps) + return false; + + return originUri.Host is "localhost" or "127.0.0.1" or "::1"; + } + } + + internal static class HttpListenerResponseExtensions + { + public static void WriteEmptyAndClose(this HttpListenerResponse response, HttpStatusCode status, string sessionId) + { + response.StatusCode = (int)status; + response.ContentLength64 = 0; + response.WithMcpHeaders(sessionId) + .Close(); + } + + public static HttpListenerResponse WithMcpHeaders(this HttpListenerResponse response, string sessionId) + { + response.AddHeader("Mcp-Session-Id", sessionId); + response.AddHeader("MCP-Protocol-Version", McpJsonRpcDispatcher.PROTOCOL_VERSION); + return response; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta new file mode 100644 index 00000000000..6609b181173 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpHttpServer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 4767a6ddc27e9455fbb1a982904fa578 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs new file mode 100644 index 00000000000..5dbdb5fa72a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs @@ -0,0 +1,193 @@ +using Cysharp.Threading.Tasks; +using DCL.Diagnostics; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using Utility.Multithreading; + +namespace DCL.McpServer.Core +{ + /// + /// Routes JSON-RPC 2.0 messages of the MCP Streamable HTTP transport to the tool registry. + /// Only the tools capability is implemented; resources and prompts are not declared. + /// + public class McpJsonRpcDispatcher + { + /// Version of the MCP specification this server implements, declared in the initialize handshake. + public const string PROTOCOL_VERSION = "2025-06-18"; + + private const string SERVER_NAME = "dcl-unity-explorer"; + + private const int PARSE_ERROR = -32700; + private const int INVALID_REQUEST = -32600; + private const int METHOD_NOT_FOUND = -32601; + private const int INVALID_PARAMS = -32602; + + // A hung tool (e.g. an asset promise that never resolves) would otherwise hold its HttpListenerContext + // open until the server stops. Cap each call so one stuck tool can't tie up the agent's connection. + private static readonly TimeSpan TOOL_CALL_TIMEOUT = TimeSpan.FromSeconds(30); + + private readonly McpToolsRegistry tools; + private readonly string serverVersion; + + // Lets an agent orchestrating several Explorer instances confirm which process answers on this port. + private readonly int processId = System.Diagnostics.Process.GetCurrentProcess().Id; + + public McpJsonRpcDispatcher(McpToolsRegistry tools, string serverVersion) + { + this.tools = tools; + this.serverVersion = serverVersion; + } + + /// + /// Returns the serialized JSON-RPC response, or null when the message is a notification + /// (or a response relayed by the client) and no reply must be sent. + /// + public async UniTask DispatchAsync(string requestJson, CancellationToken ct) + { + if (ParseRoutableRequest(requestJson, out string? earlyResponse) is not { } routable) + return earlyResponse; + + var (id, method, callParams) = routable; + + return method switch + { + "initialize" => JsonRpcEnvelope.Result(id, InitializeResult(callParams)), + "ping" => JsonRpcEnvelope.Result(id, new JObject()), + "tools/list" => JsonRpcEnvelope.Result(id, tools.ToolsListPayload()), + "tools/call" => await CallToolAsync(id, + toolName: callParams?["name"]?.Value(), + arguments: callParams?["arguments"] as JObject ?? new JObject(), + ct), + _ => JsonRpcEnvelope.Error(id, METHOD_NOT_FOUND, $"Method not found: {method}") + }; + } + + /// + /// Parses the raw message and returns the id, method and params to route on. + /// Returns null when there is nothing to route: then + /// carries the reply to send back (a JSON-RPC error) or null for a notification that gets no response. + /// + private static (JToken id, string method, JObject? callParams)? ParseRoutableRequest(string requestJson, out string? earlyResponse) + { + earlyResponse = null; + + JToken parsed; + try { parsed = JToken.Parse(requestJson); } + catch (JsonException) + { + earlyResponse = JsonRpcEnvelope.Error(null, PARSE_ERROR, "Parse error"); + return null; + } + + // -32700 is reserved for unparseable JSON; well-formed JSON that is not a JSON-RPC object + // (a bare array, number or string) is a valid document but an invalid request: -32600. + if (parsed is not JObject request) + { + earlyResponse = JsonRpcEnvelope.Error(null, INVALID_REQUEST, "Invalid request: expected a JSON-RPC object"); + return null; + } + + JToken? id = request["id"]; + string? method = request["method"]?.Value(); + + if (string.IsNullOrEmpty(method)) + { + earlyResponse = id == null ? null : JsonRpcEnvelope.Error(id, INVALID_REQUEST, "Invalid request: missing method"); + return null; + } + + // Messages without an id are notifications ("notifications/initialized" et al.) and get no response. + if (id == null) + return null; + + return (id, method, request["params"] as JObject); + } + + private JObject InitializeResult(JObject? initializeParams) + { + // We implement a single MCP revision. Per the handshake rules we answer with our version regardless, + // but a client pinned to a different revision may abort on a strict mismatch — surface it so the + // otherwise-silent interop failure is diagnosable from the server logs. + string? requestedVersion = initializeParams?["protocolVersion"]?.Value(); + + if (!string.IsNullOrEmpty(requestedVersion) && requestedVersion != PROTOCOL_VERSION) + ReportHub.LogWarning(ReportCategory.MCP, $"MCP client requested protocol version '{requestedVersion}', server responding with '{PROTOCOL_VERSION}'"); + + return new JObject + { + ["protocolVersion"] = PROTOCOL_VERSION, + ["capabilities"] = new JObject { ["tools"] = new JObject() }, + ["serverInfo"] = new JObject + { + ["name"] = SERVER_NAME, + ["version"] = serverVersion, + ["pid"] = processId, + }, + }; + } + + private async UniTask CallToolAsync(JToken id, string? toolName, JObject arguments, CancellationToken ct) + { + if (!tools.TryGet(toolName, out McpTool? tool)) + return JsonRpcEnvelope.Error(id, INVALID_PARAMS, $"Unknown tool: {toolName ?? ""}"); + + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct); + timeout.CancelAfter(TOOL_CALL_TIMEOUT); + + McpToolResult result; + + try + { + // Put the tool on the thread it declares. Switching under the timeout token turns a stalled + // main thread into the regular tool timeout instead of an indefinitely parked request. + if (tool.RequiresMainThread) + await UniTask.SwitchToMainThread(timeout.Token); + + result = await tool.ExecuteAsync(arguments, timeout.Token); + } + catch (OperationCanceledException) when (timeout.IsCancellationRequested && !ct.IsCancellationRequested) + { + ReportHub.LogWarning(ReportCategory.MCP, $"Tool '{toolName}' timed out after {TOOL_CALL_TIMEOUT.TotalSeconds:0}s"); + result = McpToolResult.Error($"Tool '{toolName}' timed out after {TOOL_CALL_TIMEOUT.TotalSeconds:0}s"); + } + catch (OperationCanceledException) { throw; } + catch (Exception e) + { + ReportHub.LogException(e, ReportCategory.MCP); + result = McpToolResult.Error($"Tool '{toolName}' failed: {e.Message}"); + } + + // A main-thread tool completes on the main thread and UniTask continuations run inline on the completing thread; + // undo the switch above so the envelope serialization below never spends main-thread time. + if (PlayerLoopHelper.IsMainThread) + await DCLTask.SwitchToThreadPool(); + + return JsonRpcEnvelope.Result(id, result.Payload); + } + + private static class JsonRpcEnvelope + { + public static string Result(JToken id, JToken result) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["result"] = result, + }.ToString(Formatting.None); + + public static string Error(JToken? id, int code, string message) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id ?? JValue.CreateNull(), + ["error"] = new JObject + { + ["code"] = code, + ["message"] = message, + }, + }.ToString(Formatting.None); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta new file mode 100644 index 00000000000..112c4546ada --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonRpcDispatcher.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: ade5e0f3aea214bd593ed7be6eaaaee9 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs new file mode 100644 index 00000000000..568aa973dbc --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs @@ -0,0 +1,123 @@ +using Newtonsoft.Json.Linq; +using System; + +namespace DCL.McpServer.Core +{ + /// + /// Fluent builder for a tool's input or output JSON Schema. Declaring each field through a typed method keeps + /// the schema typo-safe — a mistyped type name can't slip through the way it could in a raw JSON string — and + /// produces the same { type: object, properties, required } shape a tools/list entry expects. Nested objects, + /// integer arrays and nullable fields extend the same style to the richer shapes an outputSchema needs. + /// + public sealed class McpJsonSchema + { + private readonly JObject properties = new (); + private readonly JArray requiredNames = new (); + + private McpJsonSchema() { } + + /// Starts a schema describing an object; chain the field methods and finish with . + public static McpJsonSchema Object() => + new (); + + public McpJsonSchema String(string name, string? description = null, string[]? enumValues = null, bool isRequired = false, bool nullable = false) => + Property(name, "string", description, enumValues, isRequired, nullable); + + /// + /// Adds a string field constrained to the wire names of 's members (see + /// ), so the schema and the parsing of the argument share the enum as + /// the single source of truth. narrows the choices to a subset of members. + /// + public McpJsonSchema Enum(string name, string? description = null, T[]? allowed = null, bool isRequired = false) where T : struct, Enum => + Property(name, "string", description, allowed == null ? McpWireEnum.WIRE_NAMES : McpWireEnum.WireNamesOf(allowed), isRequired, false); + + public McpJsonSchema Number(string name, string? description = null, bool isRequired = false, bool nullable = false) => + Property(name, "number", description, null, isRequired, nullable); + + public McpJsonSchema Integer(string name, string? description = null, bool isRequired = false, bool nullable = false) => + Property(name, "integer", description, null, isRequired, nullable); + + public McpJsonSchema Boolean(string name, string? description = null, bool isRequired = false, bool nullable = false) => + Property(name, "boolean", description, null, isRequired, nullable); + + /// + /// Adds a nested object field described by its own builder. A + /// field admits null in place of the object (JSON Schema "type": ["object", "null"]). + /// + public McpJsonSchema Object(string name, McpJsonSchema schema, string? description = null, bool isRequired = false, bool nullable = false) + { + JObject field = schema.Build(); + + if (nullable) + field["type"] = TypeToken("object", true); + + if (description != null) + field["description"] = description; + + return AddField(name, field, isRequired); + } + + /// Adds an array field whose items are all integers. + public McpJsonSchema IntegerArray(string name, string? description = null, bool isRequired = false) + { + var field = new JObject + { + ["type"] = "array", + ["items"] = new JObject { ["type"] = "integer" }, + }; + + if (description != null) + field["description"] = description; + + return AddField(name, field, isRequired); + } + + /// Materializes the accumulated fields into the JSON Schema object. + public JObject Build() + { + var schema = new JObject + { + ["type"] = "object", + ["properties"] = properties, + }; + + if (requiredNames.Count > 0) + schema["required"] = requiredNames; + + return schema; + } + + private McpJsonSchema Property(string name, string type, string? description, string[]? enumValues, bool isRequired, bool nullable) + { + var field = new JObject { ["type"] = TypeToken(type, nullable) }; + + if (description != null) + field["description"] = description; + + if (enumValues != null) + { + var values = new JArray(); + + foreach (string value in enumValues) + values.Add(value); + + field["enum"] = values; + } + + return AddField(name, field, isRequired); + } + + private static JToken TypeToken(string type, bool nullable) => + nullable ? new JArray { type, "null" } : type; + + private McpJsonSchema AddField(string name, JObject field, bool isRequired) + { + properties[name] = field; + + if (isRequired) + requiredNames.Add(name); + + return this; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta new file mode 100644 index 00000000000..abfcab1d96b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpJsonSchema.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 05bd14397a93a8242be559c114683862 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpRequest.cs b/Explorer/Assets/DCL/McpServer/Core/McpRequest.cs new file mode 100644 index 00000000000..dc212259801 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpRequest.cs @@ -0,0 +1,68 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using Utility.Arch; + +namespace DCL.McpServer.Core +{ + /// + /// A request component written onto an entity by an MCP tool and fulfilled by a consuming system. + /// The shared install/complete/abandon choreography lives in . + /// + public interface IMcpEcsRequest + { + /// + /// Completed exactly once: with the outcome by the fulfilling system, or with a preemption result + /// when a newer request replaces this one before it is fulfilled. + /// + UniTaskCompletionSource? Completion { get; set; } + } + + /// + /// The request/response choreography shared by MCP intent components: a tool installs the request + /// (preempting a pending one), the consuming system completes and removes it, and the tool-side timeout + /// abandons a request the simulation never picked up. + /// + public static class McpRequest + { + /// + /// Installs the request on the entity and returns the task its consuming system will complete. + /// A pending request of the same type is preempted: its awaiter is released with + /// before the component is replaced. Main thread only. + /// + public static UniTask SendAsync(World world, Entity entity, TIntent request, TResult preemptedResult) + where TIntent : struct, IMcpEcsRequest + { + if (world.TryGet(entity, out TIntent existing)) + existing.Completion?.TrySetResult(preemptedResult); + + var completion = new UniTaskCompletionSource(); + request.Completion = completion; + world.AddOrSet(entity, request); + + return completion.Task; + } + + /// + /// Removes the request component, then releases its awaiter with . + /// Takes the request by copy and removes before completing, so the awaiter's continuation observes + /// the entity without the component; the caller must be done with any refs into the entity, as the + /// removal is a structural change. + /// + public static void CompleteAndRemove(World world, Entity entity, TIntent request, TResult result) + where TIntent : struct, IMcpEcsRequest + { + UniTaskCompletionSource? completion = request.Completion; + world.Remove(entity); + completion?.TrySetResult(result); + } + + /// Drops a request the simulation never completed (tool-side timeout). Safe to call from any thread. + public static async UniTask AbandonAsync(World world, Entity entity) where TIntent : struct + { + await UniTask.SwitchToMainThread(); + + if (world.Has(entity)) + world.Remove(entity); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpRequest.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpRequest.cs.meta new file mode 100644 index 00000000000..3942b28e893 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpRequest.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0d7f764120d4efc0fed964f334432293 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpTool.cs b/Explorer/Assets/DCL/McpServer/Core/McpTool.cs new file mode 100644 index 00000000000..bfd3e8089a6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpTool.cs @@ -0,0 +1,59 @@ +using Cysharp.Threading.Tasks; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Core +{ + /// + /// A single MCP tool exposed to connected coding agents via tools/list and tools/call. + /// The abstract members are the required contract; the virtual ones (, + /// , ) are optional overrides whose defaults + /// describe a tool that takes no arguments, returns unstructured text, runs on the main thread. + /// + public abstract class McpTool + { + public abstract string Name { get; } + + public abstract string Description { get; } + + /// + /// JSON Schema of the tool arguments, surfaced as inputSchema in tools/list. Assembled from + /// , so it is a valid JSON Schema object (type=object) by construction. + /// + public JObject InputSchema => DescribeInput(McpJsonSchema.Object()).Build(); + + /// + /// Behaviour hints (read-only, destructive, idempotent, open-world) surfaced in tools/list. + /// + public abstract McpToolAnnotations Annotations { get; } + + /// + /// Optional override: JSON Schema of this tool's structuredContent, surfaced as outputSchema in + /// tools/list. Build it with . Null (the default) when the tool returns + /// only unstructured text; tools that emit override + /// this to describe that payload. + /// + public virtual JObject? OutputSchema => null; + + /// + /// Optional override: false declares the tool safe to run on a thread-pool thread, so the dispatcher + /// skips the main-thread switch and the tool answers even while the main thread is busy or paused. + /// Only for tools that touch neither ECS nor Unity state and read exclusively thread-safe sources. + /// + public virtual bool RequiresMainThread => true; + + /// + /// The tool body. Invoked on the thread declares — the main thread + /// by default, so implementations may touch ECS and Unity state directly. Offload heavy CPU work to + /// the thread pool yourself and hop back before touching that state again. Expected failures are + /// reported through , not exceptions. + /// + public abstract UniTask ExecuteAsync(JObject arguments, CancellationToken ct); + + /// + /// Optional override: declares the tool's argument fields on the provided builder. The default + /// describes a tool without arguments (an empty object schema). + /// + protected virtual McpJsonSchema DescribeInput(McpJsonSchema schema) => schema; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpTool.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpTool.cs.meta new file mode 100644 index 00000000000..f41607472fd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e258f7df9608348d98dffe25a3ae3bb6 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs new file mode 100644 index 00000000000..e5dc37d9b17 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs @@ -0,0 +1,54 @@ +using Newtonsoft.Json.Linq; + +namespace DCL.McpServer.Core +{ + /// + /// Behaviour hints for a tool (MCP spec 2025-06-18) surfaced in tools/list so an agent can reason + /// about a tool before calling it. Only hints set through the factory methods are emitted; the rest + /// are omitted rather than defaulted, because the spec's implicit defaults are surprising + /// (destructiveHint and openWorldHint default to true, readOnlyHint to false). + /// + public readonly struct McpToolAnnotations + { + private readonly bool? readOnlyHint; + private readonly bool? destructiveHint; + private readonly bool? idempotentHint; + private readonly bool? openWorldHint; + + private McpToolAnnotations(bool? readOnlyHint, bool? destructiveHint, bool? idempotentHint, bool? openWorldHint) + { + this.readOnlyHint = readOnlyHint; + this.destructiveHint = destructiveHint; + this.idempotentHint = idempotentHint; + this.openWorldHint = openWorldHint; + } + + /// + /// A tool that only reads state. The spec ignores destructiveHint/idempotentHint when readOnlyHint + /// is true, so they are left unset. stays false for the local Explorer. + /// + public static McpToolAnnotations ReadOnly(bool openWorld = false) => + new (readOnlyHint: true, destructiveHint: null, idempotentHint: null, openWorldHint: openWorld); + + /// + /// A tool that changes state. flags irreversible or data-losing + /// effects; flags that repeating the call with the same arguments + /// has no additional effect. stays false for the local Explorer. + /// + public static McpToolAnnotations Mutating(bool destructive, bool idempotent, bool openWorld = false) => + new (readOnlyHint: false, destructiveHint: destructive, idempotentHint: idempotent, openWorldHint: openWorld); + + /// Serializes the set hints to the MCP annotations object embedded in a tools/list entry. + public JObject ToJObject() + { + var json = new JObject(); + + if (readOnlyHint.HasValue) json["readOnlyHint"] = readOnlyHint.Value; + if (destructiveHint.HasValue) json["destructiveHint"] = destructiveHint.Value; + if (idempotentHint.HasValue) json["idempotentHint"] = idempotentHint.Value; + if (openWorldHint.HasValue) json["openWorldHint"] = openWorldHint.Value; + + return json; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta new file mode 100644 index 00000000000..41c1e9b50ed --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolAnnotations.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 6d6f99687071d4842921cf17924e34b2 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs new file mode 100644 index 00000000000..a2dd0615e94 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs @@ -0,0 +1,82 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; + +namespace DCL.McpServer.Core +{ + /// + /// The payload of a tools/call result: a content array of text/image items, + /// with expected failures flagged via isError instead of JSON-RPC errors. + /// + public readonly struct McpToolResult + { + public readonly JObject Payload; + + private McpToolResult(JObject payload) + { + Payload = payload; + } + + public static McpToolResult Text(string text) => + new (new JObject + { + ["content"] = new JArray { TextItem(text) }, + }); + + /// + /// A text result whose body is rendered as indented JSON. Centralizes the + /// Formatting.Indented serialization so tools never render JSON differently or forget the formatting. + /// + public static McpToolResult Json(JObject payload) => + Text(payload.ToString(Formatting.Indented)); + + /// + /// A result that surfaces both as structuredContent (validated against the + /// tool's outputSchema) and as its indented-JSON text duplicate — the spec requires the text mirror so + /// clients without structured support still read the result. This is the common path; it serializes the + /// mirror itself so the two copies cannot drift. + /// + public static McpToolResult JsonWithStructured(JObject structured) => + TextWithStructured(structured.ToString(Formatting.Indented), structured); + + /// + /// Like but with an explicit mirror, for the rare + /// case where the human-readable text is deliberately not the raw serialization of . + /// + public static McpToolResult TextWithStructured(string text, JObject structured) => + new (new JObject + { + ["content"] = new JArray { TextItem(text) }, + ["structuredContent"] = structured, + }); + + public static McpToolResult Error(string message) => + new (new JObject + { + ["content"] = new JArray { TextItem(message) }, + ["isError"] = true, + }); + + public static McpToolResult Image(byte[] imageBytes, string mimeType, string caption) => + new (new JObject + { + ["content"] = new JArray + { + new JObject + { + ["type"] = "image", + ["data"] = Convert.ToBase64String(imageBytes), + ["mimeType"] = mimeType, + }, + TextItem(caption), + }, + }); + + private static JObject TextItem(string text) => + new () + { + ["type"] = "text", + ["text"] = text, + }; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta new file mode 100644 index 00000000000..075d762862a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolResult.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 35469ba78d1a54f148c476efbc3b062f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs new file mode 100644 index 00000000000..d429c7d837e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs @@ -0,0 +1,67 @@ +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; + +namespace DCL.McpServer.Core +{ + public class McpToolsRegistry + { + private readonly Dictionary tools = new (); + private string toolsListJson = null!; + + /// + /// The tools/list payload, serialized once at Build(). Each dispatch wraps the shared immutable + /// JSON string in a fresh JRaw, so there is no shared JToken tree to clone or re-parent when + /// concurrent responses serialize on the thread pool. + /// + public JRaw ToolsListPayload() => new (toolsListJson); + + public McpToolsRegistry Add(McpTool tool) + { + tools.Add(tool.Name, tool); + return this; + } + + public McpToolsRegistry Build() + { + var toolsArray = new JArray(); + + foreach (McpTool tool in tools.Values) + { + if (tool.OutputSchema != null && !IsObjectSchema(tool.OutputSchema)) + throw new InvalidOperationException($"MCP tool '{tool.Name}' produced an invalid output schema: expected a JSON Schema object (\"type\": \"object\"). Build it with McpJsonSchema."); + + var entry = new JObject + { + ["name"] = tool.Name, + ["description"] = tool.Description, + ["inputSchema"] = tool.InputSchema, + ["annotations"] = tool.Annotations.ToJObject(), + }; + + if (tool.OutputSchema != null) + entry["outputSchema"] = tool.OutputSchema; + + toolsArray.Add(entry); + } + + toolsListJson = new JObject { ["tools"] = toolsArray }.ToString(Formatting.None); + return this; + } + + private static bool IsObjectSchema(JObject schema) => + schema["type"]?.Value() == "object"; + + public bool TryGet(string? name, [NotNullWhen(true)] out McpTool? tool) + { + tool = null; + + if (string.IsNullOrEmpty(name)) + return false; + + return tools.TryGetValue(name, out tool); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta new file mode 100644 index 00000000000..d9966b0e97c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpToolsRegistry.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3bf2f57f26a5e44fd86d2b7f1201d27e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Core/McpWireEnum.cs b/Explorer/Assets/DCL/McpServer/Core/McpWireEnum.cs new file mode 100644 index 00000000000..284a7ec990f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpWireEnum.cs @@ -0,0 +1,77 @@ +using System; +using System.Collections.Generic; +using System.Text; + +namespace DCL.McpServer.Core +{ + /// + /// Per-enum cache of wire (snake_case) member names, so a tool's schema, its argument parsing and its + /// output all derive from the enum itself: "FirstPerson" ↔ "first_person", "WALK" ↔ "walk". Built once + /// per type on first touch; parsing and formatting allocate nothing afterwards. Renaming an enum member + /// therefore renames its wire value — connected agents re-read the schema every session, but skill + /// recipes that spell a value out (.claude/skills/mcp-scene-iteration/) must be kept in sync. + /// + public static class McpWireEnum where T : struct, Enum + { + // A distinct static cache per enum type T is the intended behavior of this class. + // ReSharper disable StaticMemberInGenericType + + /// Wire names of all members, in declaration order. + public static readonly string[] WIRE_NAMES; + + private static readonly T[] VALUES; + private static readonly Dictionary BY_WIRE_NAME; + + // ReSharper restore StaticMemberInGenericType + + static McpWireEnum() + { + string[] names = Enum.GetNames(typeof(T)); + VALUES = (T[])Enum.GetValues(typeof(T)); + WIRE_NAMES = new string[names.Length]; + BY_WIRE_NAME = new Dictionary(names.Length); + + for (var i = 0; i < names.Length; i++) + { + WIRE_NAMES[i] = ToSnakeCase(names[i]); + BY_WIRE_NAME[WIRE_NAMES[i]] = VALUES[i]; + } + } + + public static bool TryParse(string wireName, out T value) => + BY_WIRE_NAME.TryGetValue(wireName, out value); + + public static string ToWire(T value) => + WIRE_NAMES[Array.IndexOf(VALUES, value)]; + + /// Wire names of a subset of members, for schemas that expose only part of the enum. Allocates the array. + public static string[] WireNamesOf(T[] members) + { + var result = new string[members.Length]; + + for (var i = 0; i < members.Length; i++) + result[i] = ToWire(members[i]); + + return result; + } + + /// "FirstPerson" → "first_person", "DroneView" → "drone_view", "SDKCamera" → "sdk_camera", "WAIT_TICK" → "wait_tick". + private static string ToSnakeCase(string memberName) + { + var builder = new StringBuilder(memberName.Length + 4); + + for (var i = 0; i < memberName.Length; i++) + { + char current = memberName[i]; + + if (char.IsUpper(current) && i > 0 && memberName[i - 1] != '_' + && (!char.IsUpper(memberName[i - 1]) || (i + 1 < memberName.Length && char.IsLower(memberName[i + 1])))) + builder.Append('_'); + + builder.Append(char.ToLowerInvariant(current)); + } + + return builder.ToString(); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Core/McpWireEnum.cs.meta b/Explorer/Assets/DCL/McpServer/Core/McpWireEnum.cs.meta new file mode 100644 index 00000000000..6620390c64c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Core/McpWireEnum.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 6a0cf69bc5be8574ba5fd4fdb4511d92 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef new file mode 100644 index 00000000000..92b00af768d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef @@ -0,0 +1,32 @@ +{ + "name": "DCL.McpServer", + "rootNamespace": "", + "references": [ + "GUID:d832748739a186646b8656bdbd447ad0", + "GUID:fa7b3fdbb04d67549916da7bd2af58ab", + "GUID:0b3eab7834a09c24ca4e84fe0d8a43ce", + "GUID:4794e238ed0f65142a4aea5848b513e5", + "GUID:1d2c76eb8b48e0b40940e8b31a679ce1", + "GUID:c80c82a8f4e04453b85fbab973d6774a", + "GUID:54d33bbd50a28174e8ba0110106203c2", + "GUID:f56000518aba31544aa96f4739e50a64", + "GUID:f3634757d00dab2429c6c11e69404e97", + "GUID:286980af24684da6acc1caa413039811", + "GUID:809870cbfd80a7b46bcf72b178ab210b", + "GUID:1b8e1e1bd01505f478f0369c04a4fb2f", + "GUID:571dc9f8bded0034f98595106462e3d0", + "GUID:0df5180c0c3a0594fbfa11f83736de9f", + "GUID:3c7b57a14671040bd8c549056adc04f5", + "GUID:f51ebe6a0ceec4240a699833d6309b23", + "GUID:e25ef972de004615a22937e739de2def" + ], + "includePlatforms": [], + "excludePlatforms": [], + "allowUnsafeCode": false, + "overrideReferences": false, + "precompiledReferences": [], + "autoReferenced": true, + "defineConstraints": [], + "versionDefines": [], + "noEngineReferences": false +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta new file mode 100644 index 00000000000..092bf9535d9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/DCL.McpServer.asmdef.meta @@ -0,0 +1,3 @@ +fileFormatVersion: 2 +guid: f254b2009e854709a0ef0022a7ca697c +timeCreated: 1784219963 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems.meta b/Explorer/Assets/DCL/McpServer/Systems.meta new file mode 100644 index 00000000000..b7964b54fef --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: c6ba4ac038d1a4f32b295c8fe24fd984 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref new file mode 100644 index 00000000000..b24e4ef9e3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref @@ -0,0 +1,3 @@ +{ + "reference": "GUID:fc4fd35fb877e904d8cedee73b2256f6" +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta new file mode 100644 index 00000000000..7d2760a40bd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/DCL.McpServer.asmref.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: 38580450dd18841eb82d84648bb3694e +AssemblyDefinitionReferenceImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs new file mode 100644 index 00000000000..200e7e41872 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs @@ -0,0 +1,82 @@ +using Arch.Core; +using Arch.SystemGroups; +using Cysharp.Threading.Tasks; +using DCL.Character.CharacterMotion.Components; +using DCL.CharacterMotion.Components; +using DCL.CharacterMotion.Systems; +using DCL.Diagnostics; +using DCL.Input; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using ECS.Abstract; +using UnityEngine; + +namespace DCL.McpServer.Systems +{ + /// + /// While an is present on the player entity, re-asserts its axes into + /// after the real-input systems have written it, so an agent-requested + /// walk survives the per-frame overwrite performed by . + /// + [UpdateInGroup(typeof(InputGroup))] + [UpdateAfter(typeof(UpdateInputMovementSystem))] + [UpdateAfter(typeof(UpdateInputJumpSystem))] + [LogCategory(ReportCategory.MCP)] + public partial class McpInputOverrideSystem : BaseUnityLoopSystem + { + private readonly Entity playerEntity; + + private SingleInstanceEntity physicsTick; + + internal McpInputOverrideSystem(World world, Entity playerEntity) : base(world) + { + this.playerEntity = playerEntity; + } + + public override void Initialize() + { + base.Initialize(); + physicsTick = World.CachePhysicsTick(); + } + + protected override void Update(float t) + { + ref McpEcsMovementOverride ecsMovementOverride = ref World.TryGetRef(playerEntity, out bool overrideExists); + + if (!overrideExists) + return; + + ref MovementInputComponent movement = ref World.TryGetRef(playerEntity, out bool hasMovement); + + if (UnityEngine.Time.time < ecsMovementOverride.EndTime) + { + if (hasMovement) + { + movement.Axes = ecsMovementOverride.Axes; + movement.Kind = ecsMovementOverride.Kind; + } + + if (ecsMovementOverride.JumpRequested) + { + ecsMovementOverride.JumpRequested = false; + + ref JumpInputComponent jump = ref World.TryGetRef(playerEntity, out bool hasJump); + + if (hasJump) + jump.Trigger.TickWhenJumpOccurred = physicsTick.GetPhysicsTickComponent(World).Tick + 1; + } + } + else + { + if (hasMovement) + { + movement.Axes = Vector2.zero; + movement.Kind = MovementKind.IDLE; + } + + // The override is copied out before the structural removal; no component refs are touched afterwards. + McpRequest.CompleteAndRemove(World, playerEntity, ecsMovementOverride, AsyncUnit.Default); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta new file mode 100644 index 00000000000..6d1c1b9ca8c --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpInputOverrideSystem.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1d8aa7853be9e41d6920f15f5468914b \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpPointerEventSystem.cs b/Explorer/Assets/DCL/McpServer/Systems/McpPointerEventSystem.cs new file mode 100644 index 00000000000..0fbc1566633 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpPointerEventSystem.cs @@ -0,0 +1,361 @@ +using Arch.Core; +using Arch.SystemGroups; +using Arch.SystemGroups.DefaultSystemGroups; +using CRDT; +using CrdtEcsBridge.Physics; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.Diagnostics; +using DCL.ECSComponents; +using DCL.Interaction.PlayerOriginated.Components; +using DCL.Interaction.PlayerOriginated.Utility; +using DCL.Interaction.Systems; +using DCL.Interaction.Utility; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using ECS.Abstract; +using ECS.SceneLifeCycle; +using ECS.Unity.PrimitiveColliders.Components; +using ECS.Unity.Transforms.Components; +using SceneRunner.Scene; +using UnityEngine; +using RaycastHit = UnityEngine.RaycastHit; + +namespace DCL.McpServer.Systems +{ + /// + /// + /// Delivers a single agent-requested pointer event to a scene entity while a + /// is present on the player entity. The aim is validated with the + /// same physics raycast the reticle pipeline uses (camera origin, , + /// occlusion and max-distance rules apply), then the target's + /// is filled exactly as fills it for a real press, so the + /// unmodified scene-world write-back emits an identical PBPointerEventsResult. + /// + /// + /// A release that follows a press () is delivered only once + /// the scene has advanced past the press tick and only to the world that received the press; the + /// click_entity tool composes a full click from two such intents. + /// + /// + /// Runs after so its per-frame intent Initialize cannot wipe + /// the synthetic event before the scene-world flush, which happens later in the same frame. + /// + /// + [UpdateInGroup(typeof(PresentationSystemGroup))] + [UpdateAfter(typeof(ProcessPointerEventsSystem))] + [LogCategory(ReportCategory.MCP)] + public partial class McpPointerEventSystem : BaseUnityLoopSystem + { + private const float MAX_RAYCAST_DISTANCE = 100f; + private const float AIM_DIRECTION_EPSILON_SQR = 0.0001f; + + private static readonly QueryDescription ALL_ENTITIES = new (); + + private readonly IScenesCache scenesCache; + private readonly IEntityCollidersGlobalCache collidersGlobalCache; + private readonly Entity playerEntity; + + private SingleInstanceEntity playerCamera; + + internal McpPointerEventSystem(World world, + IScenesCache scenesCache, + IEntityCollidersGlobalCache collidersGlobalCache, + Entity playerEntity) : base(world) + { + this.scenesCache = scenesCache; + this.collidersGlobalCache = collidersGlobalCache; + this.playerEntity = playerEntity; + } + + public override void Initialize() + { + base.Initialize(); + playerCamera = World.CacheCamera(); + } + + protected override void Update(float t) + { + ref McpEcsPointerEventIntent intent = ref World.TryGetRef(playerEntity, out bool exists); + + if (!exists) + return; + + ISceneFacade? scene = scenesCache.CurrentScene.Value; + + if (scene == null || !scene.SceneStateProvider.IsCurrent || scene.SceneStateProvider.IsNotRunningState()) + { + CompleteAndRemove(in intent, Failure(in intent, "no running current scene to deliver the pointer event to")); + return; + } + + World sceneWorld = scene.EcsExecutor.World; + uint tick = scene.SceneStateProvider.TickNumber; + + if (intent.Press.HasValue) + { + McpPressHandoff press = intent.Press.Value; + + // A mid-click reload swaps in a new world for the same parcel; the press handoff belongs to the + // disposed one (entity ids get recycled), so the release can only be failed. + if (!ReferenceEquals(sceneWorld, press.World)) + { + CompleteAndRemove(in intent, Failure(in intent, "the scene reloaded mid-click; only the press may have been delivered")); + return; + } + + // The scene must observe the press on an earlier tick than the release, otherwise ordering is ambiguous. + if (tick <= press.Tick) + return; + + DeliverRelease(in intent, in press, sceneWorld, tick, out McpPointerClickResult releaseResult); + CompleteAndRemove(in intent, releaseResult); + return; + } + + TryDeliver(in intent, sceneWorld, tick, null, out McpPointerClickResult result); + CompleteAndRemove(in intent, result); + } + + /// The intent is copied out before the structural removal, so the caller's ref must not be touched afterwards. + private void CompleteAndRemove(in McpEcsPointerEventIntent intent, McpPointerClickResult result) => + McpRequest.CompleteAndRemove(World, playerEntity, intent, result); + + private static McpPointerClickResult Failure(in McpEcsPointerEventIntent intent, string reason) => + new () + { + Hit = false, + FailureReason = reason, + SceneEntityId = intent.TargetEntityId, + }; + + private bool TryDeliver(in McpEcsPointerEventIntent intent, World sceneWorld, uint tick, Entity? pressEntity, out McpPointerClickResult result) + { + if (!TryResolveTarget(in intent, sceneWorld, pressEntity, out Entity targetEntity, out result)) + return false; + + bool requireTarget = intent.TargetEntityId >= 0; + + Vector3 aimPoint = intent.AimPoint ?? ResolveEntityAimPoint(sceneWorld, targetEntity); + + CameraComponent camera = playerCamera.GetCameraComponent(World); + Vector3 origin = camera.Camera.transform.position; + Vector3 direction = aimPoint - origin; + + if (direction.sqrMagnitude < AIM_DIRECTION_EPSILON_SQR) + { + result = Failure(in intent, "the camera is on top of the aim point; move back and retry"); + return false; + } + + var ray = new Ray(origin, direction.normalized); + + if (!Physics.Raycast(ray, out RaycastHit hit, MAX_RAYCAST_DISTANCE, PhysicsLayers.PLAYER_ORIGIN_RAYCAST_MASK)) + { + result = Failure(in intent, "the ray from the camera hit nothing (target may lack a collider)"); + return false; + } + + if (!collidersGlobalCache.TryGetSceneEntity(hit.collider, out GlobalColliderSceneEntityInfo hitInfo) + || !ReferenceEquals(hitInfo.EcsExecutor.World, sceneWorld)) + { + result = Failure(in intent, $"the ray hit a non-scene collider '{hit.collider.name}'"); + return false; + } + + Entity hitEntity = hitInfo.ColliderSceneEntityInfo.EntityReference; + + if (requireTarget && hitEntity != targetEntity) + { + result = Failure(in intent, "another collider blocks the line of sight to the target"); + result.BlockedByEntityId = hitEntity.Id; + result.BlockedByCrdtId = hitInfo.ColliderSceneEntityInfo.SDKEntity.Id; + result.BlockedByColliderName = hit.collider.name; + return false; + } + + // In pure aim-point mode the raycast decides the target. + targetEntity = hitEntity; + + if (!hitInfo.TryGetPointerEvents(out PBPointerEvents? pbPointerEvents)) + { + result = Failure(in intent, $"entity {targetEntity.Id} has no PointerEvents component (not clickable)"); + result.SceneEntityId = targetEntity.Id; + return false; + } + + if (!sceneWorld.TryGet(targetEntity, out CRDTEntity crdtEntity)) + { + result = Failure(in intent, $"entity {targetEntity.Id} has no CRDTEntity; the scene cannot receive results for it"); + result.SceneEntityId = targetEntity.Id; + return false; + } + + if (!IsQualified(pbPointerEvents!, ray, hit, camera, out float distance, out string? hoverText, out bool hasCursorEntry)) + { + result = Failure(in intent, hasCursorEntry + ? $"target is out of range for its pointer events (hit distance {distance:F2}m)" + : "the target's pointer events are proximity-type only; a cursor click cannot trigger them"); + + result.SceneEntityId = targetEntity.Id; + result.CrdtEntityId = crdtEntity.Id; + result.Distance = distance; + return false; + } + + pbPointerEvents!.AppendPointerEventResultsIntent.Initialize(hit, ray); + pbPointerEvents.AppendPointerEventResultsIntent.AddInputAction(intent.Button, intent.EventType); + + result = new McpPointerClickResult + { + Hit = true, + SceneEntityId = targetEntity.Id, + CrdtEntityId = crdtEntity.Id, + HoverText = hoverText, + HitPoint = hit.point, + Distance = distance, + Press = new McpPressHandoff + { + World = sceneWorld, + Entity = targetEntity, + Tick = tick, + Hit = hit, + Ray = ray, + }, + }; + + return true; + } + + /// + /// Delivers the release leg of a click. If the target moved out from under the ray after the press + /// (or its distance gate no longer qualifies), the press-frame hit is reused so the entity still + /// receives an ordered PetUp, and the divergence is reported via . + /// + private void DeliverRelease(in McpEcsPointerEventIntent intent, in McpPressHandoff press, World sceneWorld, uint tick, out McpPointerClickResult result) + { + if (TryDeliver(in intent, sceneWorld, tick, press.Entity, out result)) + return; + + // Fresh delivery failed: fall back to the press-frame hit if the component is still reachable. + if (sceneWorld.IsAlive(press.Entity) && sceneWorld.TryGet(press.Entity, out PBPointerEvents? pbPointerEvents) && pbPointerEvents != null) + { + pbPointerEvents.AppendPointerEventResultsIntent.Initialize(press.Hit, press.Ray); + pbPointerEvents.AppendPointerEventResultsIntent.AddInputAction(intent.Button, intent.EventType); + + result = new McpPointerClickResult + { + Hit = true, + SceneEntityId = press.Entity.Id, + UpRayMissed = true, + }; + + return; + } + + // Nothing was delivered: keep the fresh failure reason and flag that only the press landed. + result.UpRayMissed = true; + } + + private static bool TryResolveTarget(in McpEcsPointerEventIntent intent, World sceneWorld, Entity? pressEntity, out Entity resolved, out McpPointerClickResult result) + { + result = default; + resolved = Entity.Null; + + // Aim-point mode: the validation raycast picks the entity. + if (intent.AimPoint.HasValue && intent.TargetEntityId < 0) + return true; + + // The press already resolved the target; only re-validate that it is still alive. + if (pressEntity.HasValue) + { + if (sceneWorld.IsAlive(pressEntity.Value)) + { + resolved = pressEntity.Value; + return true; + } + + result = Failure(in intent, "the target entity was destroyed mid-click"); + return false; + } + + Entity found = Entity.Null; + int targetId = intent.TargetEntityId; + + sceneWorld.Query(in ALL_ENTITIES, entity => + { + if (entity.Id == targetId) + found = entity; + }); + + if (found == Entity.Null) + { + result = Failure(in intent, $"no entity with id {targetId} in the current scene world"); + return false; + } + + resolved = found; + return true; + } + + /// Aim at the collider volume when available; entity pivots can sit at hinges or bases and miss. + private static Vector3 ResolveEntityAimPoint(World sceneWorld, Entity entity) + { + if (sceneWorld.TryGet(entity, out PrimitiveColliderComponent primitiveCollider) && primitiveCollider.Collider != null) + return primitiveCollider.Collider.bounds.center; + + if (sceneWorld.TryGet(entity, out TransformComponent transformComponent) && transformComponent.Transform != null) + return transformComponent.Transform.position; + + return Vector3.zero; + } + + /// + /// Mirrors the cursor-entry qualification of : entries get their + /// defaults prepared and the distance gate is evaluated per entry, the last cursor entry winning, exactly + /// like the production loop. Also picks the hover text a real reticle hover would show for this button. + /// + private bool IsQualified(PBPointerEvents pbPointerEvents, in Ray ray, in RaycastHit hit, in CameraComponent camera, out float distance, out string? hoverText, out bool hasCursorEntry) + { + distance = camera.Mode == CameraMode.FirstPerson + ? hit.distance + : Vector3.Distance(hit.point, camera.PlayerFocus.position); + + float? playerDistance = null; + + if (World.TryGet(playerEntity, out CharacterTransform characterTransform)) + playerDistance = Vector3.Distance(hit.point, characterTransform.Position); + + var raycastResult = new PlayerOriginRaycastResultForSceneEntities(); + raycastResult.SetRay(ray); + raycastResult.SetupHit(hit, default(GlobalColliderSceneEntityInfo), distance, playerDistance); + + var isAtDistance = false; + hoverText = null; + hasCursorEntry = false; + + for (var i = 0; i < pbPointerEvents.PointerEvents!.Count; i++) + { + PBPointerEvents.Types.Entry entry = pbPointerEvents.PointerEvents[i]!; + + if (entry.InteractionType != InteractionType.Cursor) + continue; + + hasCursorEntry = true; + + PBPointerEvents.Types.Info info = entry.EventInfo!; + info.PrepareDefaultValues(); + + isAtDistance = InteractionInputUtils.IsQualifiedByDistance(in raycastResult, info); + + if (!isAtDistance) + continue; + + if (hoverText == null && info.HasHoverText && !string.IsNullOrEmpty(info.HoverText)) + hoverText = info.HoverText; + } + + return isAtDistance; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpPointerEventSystem.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpPointerEventSystem.cs.meta new file mode 100644 index 00000000000..821b204664e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpPointerEventSystem.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 45b28969f78d4a59b05fdd15adf4041b \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs new file mode 100644 index 00000000000..b09b1e74ab9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs @@ -0,0 +1,171 @@ +using Arch.SystemGroups; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.CharacterCamera; +using DCL.Chat.MessageBus; +using DCL.Diagnostics; +using DCL.Interaction.Utility; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using DCL.McpServer.Utils; +using DCL.PluginSystem.Global; +using DCL.RealmNavigation; +using DCL.UI.DebugMenu.MessageBus; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Global.AppArgs; +using SceneRunner.Debugging.Hub; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Systems +{ + /// + /// Hosts the embedded MCP server so external coding agents can observe and drive the client. + /// Registered only when the mcp/mcp-port app arg is present (command line or deep link); + /// the server binds to 127.0.0.1 exclusively and validates browser Origins. + /// + public class McpServerPlugin : IDCLGlobalPluginWithoutSettings + { + private const int DEFAULT_PORT = 8123; + + private const int MIN_PORT = 1024; + private const int MAX_PORT = 65535; + + private readonly int port; + + private readonly ICoroutineRunner coroutineRunner; + private readonly ILoadingStatus loadingStatus; + + private readonly IChatMessagesBus chatMessagesBus; + private readonly ExposedCameraData exposedCameraData; + + private readonly Arch.Core.World globalWorld; + private readonly IGlobalWorldActions globalWorldActions; + private readonly IEntityCollidersGlobalCache entityCollidersGlobalCache; + private readonly IWorldInfoHub worldInfoHub; + + private readonly IScenesCache scenesCache; + private readonly ICurrentSceneInfo currentSceneInfo; + private readonly ECSReloadScene reloadSceneController; + private readonly bool localSceneDevelopment; + + private readonly SceneLogBuffer logBuffer; + private readonly DebugMenuConsoleLogEntryBus logEntryBus; + + private McpHttpServer? server; + private CancellationTokenSource? serverCts; + + private ScreenshotTool? screenshotTool; + + public McpServerPlugin( + IAppArgs appArgs, + IGlobalWorldActions globalWorldActions, + IChatMessagesBus chatMessagesBus, + IScenesCache scenesCache, + ICurrentSceneInfo currentSceneInfo, + ILoadingStatus loadingStatus, + IWorldInfoHub worldInfoHub, + ECSReloadScene reloadSceneController, + DiagnosticsContainer diagnosticsContainer, + ExposedCameraData exposedCameraData, + IEntityCollidersGlobalCache entityCollidersGlobalCache, + ICoroutineRunner coroutineRunner, + Arch.Core.World globalWorld, + bool localSceneDevelopment) + { + port = appArgs.TryGetValue(AppArgsFlags.MCP_PORT, out string? portValue) + && int.TryParse(portValue, out int parsedPort) + && parsedPort is >= MIN_PORT and <= MAX_PORT + ? parsedPort + : DEFAULT_PORT; + + this.globalWorldActions = globalWorldActions; + this.chatMessagesBus = chatMessagesBus; + this.scenesCache = scenesCache; + this.currentSceneInfo = currentSceneInfo; + this.loadingStatus = loadingStatus; + this.worldInfoHub = worldInfoHub; + this.reloadSceneController = reloadSceneController; + this.exposedCameraData = exposedCameraData; + this.entityCollidersGlobalCache = entityCollidersGlobalCache; + this.coroutineRunner = coroutineRunner; + this.globalWorld = globalWorld; + this.localSceneDevelopment = localSceneDevelopment; + + logBuffer = new SceneLogBuffer(); + logEntryBus = new DebugMenuConsoleLogEntryBus(); + logEntryBus.MessageAdded += logBuffer.Append; + diagnosticsContainer.AddDebugConsoleHandler(logEntryBus); + } + + public void Dispose() + { + logEntryBus.MessageAdded -= logBuffer.Append; + screenshotTool?.Dispose(); + server?.Dispose(); + serverCts.SafeCancelAndDispose(); + } + + public void InjectToWorld(ref ArchSystemsWorldBuilder builder, in GlobalPluginArguments arguments) + { + McpInputOverrideSystem.InjectToWorld(ref builder, arguments.PlayerEntity); + McpPointerEventSystem.InjectToWorld(ref builder, scenesCache, entityCollidersGlobalCache, arguments.PlayerEntity); + + screenshotTool = new ScreenshotTool(coroutineRunner, globalWorld, arguments.PlayerEntity); + + var toolsRegistry = new McpToolsRegistry() + .Add(screenshotTool) + .Add(new GetPlayerStateTool(globalWorld, arguments.PlayerEntity, exposedCameraData, currentSceneInfo)) + .Add(new GetSceneStateTool(scenesCache, currentSceneInfo, loadingStatus, localSceneDevelopment)) + .Add(new GetSceneLogsTool(logBuffer)) + .Add(new TeleportTool(chatMessagesBus, scenesCache, loadingStatus)) + .Add(new MoveToTool(globalWorldActions, globalWorld, arguments.PlayerEntity)) + .Add(new LookAtTool(globalWorldActions, globalWorld, arguments.PlayerEntity, exposedCameraData)) + .Add(new SetCameraModeTool(globalWorld, exposedCameraData)) + .Add(new SetCameraPoseTool(globalWorld, arguments.PlayerEntity, exposedCameraData)) + .Add(new WalkTool(globalWorld, arguments.PlayerEntity)) + .Add(new SendChatTool(chatMessagesBus)) + .Add(new ReloadSceneTool(reloadSceneController, scenesCache, globalWorld, arguments.PlayerEntity, arguments.SkyboxEntity)) + .Add(new ListSceneEntitiesTool(worldInfoHub)) + .Add(new GetEntityDetailsTool(worldInfoHub)) + .Add(new TriggerEmoteTool(globalWorldActions)) + .Add(new ClickEntityTool(globalWorld, arguments.PlayerEntity)) + .Build(); + + server = new McpHttpServer(toolsRegistry, port); + serverCts = serverCts.SafeRestart(); + + bool started = server.TryStart(); + + if (started) + server.RunAsync(serverCts.Token).Forget(); + + AnnounceStatusWhenLoadedAsync(started, server.EndpointUrl, serverCts.Token).Forget(); + } + + /// + /// Reports the server address (or the startup failure) once loading completes, so the message + /// reaches the scene debug console: its UI subscribes to log entries only after this plugin runs, + /// and a line logged at server start would be dropped. + /// + private async UniTaskVoid AnnounceStatusWhenLoadedAsync(bool started, string endpointUrl, CancellationToken ct) + { + try + { + await UniTask.WaitUntil(() => loadingStatus.CurrentStage.Value == LoadingStatus.LoadingStage.Completed, cancellationToken: ct); + + if (started) + ReportHub.Log(LogType.Log, ReportCategory.MCP, $"MCP server listening on {endpointUrl}"); + else + ReportHub.LogError(ReportCategory.MCP, $"MCP server failed to start on port {port} — agent connections unavailable (pass a different --mcp-port)"); + } + catch (OperationCanceledException) + { + ReportHub.Log(LogType.Log, ReportCategory.MCP, "MCP server status announcement cancelled before loading completed"); + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta new file mode 100644 index 00000000000..9336efcf1cd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Systems/McpServerPlugin.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5d7cb65f2d6064e8f98e97c77b2ccb7e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests.meta b/Explorer/Assets/DCL/McpServer/Tests.meta new file mode 100644 index 00000000000..309d70874e3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 85f2f58b3ff94ddfa14bb5bda0658b63 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref new file mode 100644 index 00000000000..9c56917b757 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref @@ -0,0 +1,3 @@ +{ + "reference": "GUID:da80994a355e49d5b84f91c0a84a721f" +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta new file mode 100644 index 00000000000..b2e1703bffa --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/DCL.McpServer.Tests.asmref.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: b6d1d98221344ffa8441fb344268804f +AssemblyDefinitionReferenceImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs new file mode 100644 index 00000000000..28ca327be66 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs @@ -0,0 +1,62 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + /// + /// A configurable test double: its ExecuteAsync returns a preset result, + /// or throws a preset exception, and records the arguments and cancellation token it was called with. + /// Keeps the routing tests independent of the real tool implementations. + /// + internal sealed class FakeMcpTool : McpTool + { + private readonly Func execute; + + public override string Name { get; } + + public override string Description { get; } + + public override McpToolAnnotations Annotations { get; } + + public int CallCount { get; private set; } + + public JObject? LastArguments { get; private set; } + + private FakeMcpTool(string name, string description, McpToolAnnotations annotations, + Func execute) + { + Name = name; + Description = description; + Annotations = annotations; + this.execute = execute; + } + + /// A tool whose ExecuteAsync returns (defaults to a text result). + public static FakeMcpTool Returning(string name, McpToolResult? result = null, + McpToolAnnotations? annotations = null) + { + McpToolResult toReturn = result ?? McpToolResult.Text($"{name} ran"); + return new FakeMcpTool(name, $"{name} description", + annotations ?? McpToolAnnotations.ReadOnly(), (_, _) => toReturn); + } + + /// A tool whose ExecuteAsync throws . + public static FakeMcpTool Throwing(string name, Exception exception, + McpToolAnnotations? annotations = null) => + new (name, $"{name} description", + annotations ?? McpToolAnnotations.ReadOnly(), (_, _) => throw exception); + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.String("value", "Any value."); + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CallCount++; + LastArguments = arguments; + return UniTask.FromResult(execute(arguments, ct)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta new file mode 100644 index 00000000000..1d1f4a9775a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/FakeMcpTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e7a2d21be39bd5441ba0b52f68f67b34 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs new file mode 100644 index 00000000000..6ed339b06b7 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs @@ -0,0 +1,88 @@ +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class GetEntityDetailsToolShould + { + private const string CURRENT_SCENE = "CURRENT"; + + private IWorldInfoHub worldInfoHub = null!; + private IWorldInfo worldInfo = null!; + private GetEntityDetailsTool tool = null!; + + [SetUp] + public void Setup() + { + worldInfo = Substitute.For(); + worldInfoHub = Substitute.For(); + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns(worldInfo); + tool = new GetEntityDetailsTool(worldInfoHub); + } + + [Test] + public void ReturnTheDumpWholeWhenItFitsTheBudget() + { + // Arrange + const string DUMP = "Components of entity 5, total count: 1\n1) PBTransform"; + worldInfo.EntityComponentsInfo(5).Returns(DUMP); + + // Act + string text = TextOf(Execute(5)); + + // Assert + Assert.That(text, Is.EqualTo(DUMP)); + Assert.That(text, Does.Not.Contain("truncated")); + } + + [Test] + public void TruncateWithANoteWhenTheDumpExceedsTheBudget() + { + // Arrange + string dump = new ('x', 20000); + worldInfo.EntityComponentsInfo(5).Returns(dump); + + // Act + string text = TextOf(Execute(5)); + + // Assert + Assert.That(text.Length, Is.LessThan(dump.Length)); + Assert.That(text, Does.Contain($"output truncated at 8000/{dump.Length} chars")); + } + + [Test] + public void ErrorWhenNoSceneWorldIsFound() + { + // Arrange + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns((IWorldInfo?)null); + + // Act + McpToolResult result = Execute(5); + + // Assert + Assert.That(result.Payload["isError"]!.Value(), Is.True); + } + + [Test] + public void ErrorWhenEntityIdIsMissing() + { + // Act + McpToolResult result = tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + + // Assert + Assert.That(result.Payload["isError"]!.Value(), Is.True); + } + + private McpToolResult Execute(int entityId) => + tool.ExecuteAsync(new JObject { ["entityId"] = entityId }, CancellationToken.None).GetAwaiter().GetResult(); + + private static string TextOf(McpToolResult result) => + result.Payload["content"]![0]!["text"]!.Value()!; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta new file mode 100644 index 00000000000..5f8d40e23e5 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetEntityDetailsToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: c5c1a3c1b1573d04dae5c672589e81de \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs new file mode 100644 index 00000000000..b960fc84a89 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs @@ -0,0 +1,75 @@ +using Arch.Core; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tests +{ + public class GetPlayerStateToolShould + { + private World world = null!; + private GameObject playerGameObject = null!; + private GetPlayerStateTool tool = null!; + + [SetUp] + public void Setup() + { + world = World.Create(); + playerGameObject = new GameObject(nameof(GetPlayerStateToolShould)); + + Entity playerEntity = world.Create(new CharacterTransform(playerGameObject.transform)); + world.Create(new CameraComponent()); // the tool reads the camera mode through CacheCamera() + + tool = new GetPlayerStateTool(world, playerEntity, new ExposedCameraData(), Substitute.For()); + } + + [TearDown] + public void TearDown() + { + world.Dispose(); + Object.DestroyImmediate(playerGameObject); + } + + [Test] + public void DeclareAnObjectOutputSchema() + { + Assert.That(tool.OutputSchema["type"]!.Value(), Is.EqualTo("object")); + } + + [Test] + public void ModelTheAddressAsANullableString() + { + var addressType = (JArray)tool.OutputSchema["properties"]!["address"]!["type"]!; + Assert.That(addressType.ToObject(), Is.EqualTo(new[] { "string", "null" })); + } + + [Test] + public void ModelTheCameraAsANestedObject() + { + JToken camera = tool.OutputSchema["properties"]!["camera"]!; + + Assert.That(camera["type"]!.Value(), Is.EqualTo("object")); + Assert.That(camera["properties"]!["mode"]!["type"]!.Value(), Is.EqualTo("string")); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Act + var structured = (JObject)Execute().Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema, structured); + } + + private McpToolResult Execute() => + tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta new file mode 100644 index 00000000000..c474cb9df5b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetPlayerStateToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 486751286fc8bcc47af7880e22187283 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs new file mode 100644 index 00000000000..a52a94729a7 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs @@ -0,0 +1,105 @@ +using DCL.Diagnostics; +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using DCL.RealmNavigation; +using DCL.Utilities; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Tests +{ + public class GetSceneStateToolShould + { + private IScenesCache scenesCache = null!; + private ICurrentSceneInfo currentSceneInfo = null!; + private ILoadingStatus loadingStatus = null!; + private GetSceneStateTool tool = null!; + + [SetUp] + public void Setup() + { + scenesCache = Substitute.For(); + currentSceneInfo = Substitute.For(); + loadingStatus = Substitute.For(); + + scenesCache.CurrentParcel.Returns(new ReactiveProperty(new Vector2Int(1, 2))); + scenesCache.CurrentScene.Returns(new ReactiveProperty(null)); + loadingStatus.CurrentStage.Returns(new ReactiveProperty(default)); + + tool = new GetSceneStateTool(scenesCache, currentSceneInfo, loadingStatus, localSceneDevelopment: false); + } + + [Test] + public void MirrorTheStateInBothTextAndStructuredContent() + { + // Act + McpToolResult result = Execute(); + + // Assert + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured, Is.Not.Null); + Assert.That(result.Payload["content"]![0]!["text"]!.Value(), Is.EqualTo(structured.ToString(Formatting.Indented))); + } + + [Test] + public void ReportAnAbsentSceneAsAJsonNull() + { + // Act + McpToolResult result = Execute(); + + // Assert + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured["scene"]!.Type, Is.EqualTo(JTokenType.Null)); + } + + [Test] + public void DeclareAnObjectOutputSchemaThatAdmitsANullScene() + { + // Act + JObject schema = tool.OutputSchema; + + // Assert + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + + var sceneType = (JArray)schema["properties"]!["scene"]!["type"]!; + Assert.That(sceneType.ToObject(), Is.EqualTo(new[] { "object", "null" })); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Arrange — a populated scene so the nested "scene" object is covered, not just the top level. + ISceneStateProvider sceneStateProvider = Substitute.For(); + sceneStateProvider.State.Returns(new Atomic(SceneState.Running)); + + ISceneData sceneData = Substitute.For(); + sceneData.SceneLoadingConcluded.Returns(true); + + ISceneFacade scene = Substitute.For(); + scene.Info.Returns(new SceneShortInfo(new Vector2Int(1, 2), "Test scene", "7")); + scene.SceneStateProvider.Returns(sceneStateProvider); + scene.SceneData.Returns(sceneData); + scene.IsSceneReady().Returns(true); + + scenesCache.CurrentScene.Returns(new ReactiveProperty(scene)); + currentSceneInfo.SceneStatus.Returns(new ReactiveProperty(null)); + + // Act + var structured = (JObject)Execute().Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema, structured); + } + + private McpToolResult Execute() => + tool.ExecuteAsync(new JObject(), CancellationToken.None).GetAwaiter().GetResult(); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta new file mode 100644 index 00000000000..c1d5e41b5ff --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/GetSceneStateToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 523603bbeedddae44833a5a53f9793ae \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs new file mode 100644 index 00000000000..e47378dfac1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs @@ -0,0 +1,107 @@ +using DCL.McpServer.Core; +using DCL.McpServer.Tools; +using Newtonsoft.Json.Linq; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Collections.Generic; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class ListSceneEntitiesToolShould + { + private const string CURRENT_SCENE = "CURRENT"; + + private IWorldInfoHub worldInfoHub = null!; + private IWorldInfo worldInfo = null!; + private ListSceneEntitiesTool tool = null!; + + [SetUp] + public void Setup() + { + worldInfo = Substitute.For(); + worldInfoHub = Substitute.For(); + worldInfoHub.WorldInfo(CURRENT_SCENE).Returns(worldInfo); + tool = new ListSceneEntitiesTool(worldInfoHub); + } + + [Test] + public void AddAnActionableLineWhenNotEverythingIsShown() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(10)); + + // Act + string text = TextOf(Execute(limit: 3)); + + // Assert + Assert.That(text, Does.Contain("total=10 returned=3")); + Assert.That(text, Does.Contain("3 of 10 shown")); + } + + [Test] + public void OmitTheActionableLineWhenEverythingFits() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(3)); + + // Act + string text = TextOf(Execute(limit: 200)); + + // Assert + Assert.That(text, Does.Contain("total=3 returned=3")); + Assert.That(text, Does.Not.Contain("shown")); + } + + [Test] + public void MirrorTheListingInStructuredContentWhileKeepingTheText() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(10)); + + // Act + McpToolResult result = Execute(limit: 3); + + // Assert — text output is untouched + Assert.That(TextOf(result), Does.Contain("total=10 returned=3")); + + // Assert — structured mirror carries the same figures + var structured = (JObject)result.Payload["structuredContent"]!; + Assert.That(structured["total"]!.Value(), Is.EqualTo(10)); + Assert.That(structured["returned"]!.Value(), Is.EqualTo(3)); + Assert.That(structured["truncated"]!.Value(), Is.True); + Assert.That(((JArray)structured["entityIds"]!).ToObject(), Is.EqualTo(new[] { 0, 1, 2 })); + } + + [Test] + public void KeepTheOutputSchemaInSyncWithTheStructuredPayload() + { + // Arrange + worldInfo.EntityIds().Returns(Ids(3)); + + // Act + var structured = (JObject)Execute(limit: 200).Payload["structuredContent"]!; + + // Assert + McpSchemaAssert.KeysMatch(tool.OutputSchema, structured); + } + + private McpToolResult Execute(int limit) => + tool.ExecuteAsync(new JObject { ["limit"] = limit }, CancellationToken.None).GetAwaiter().GetResult(); + + private static IReadOnlyList Ids(int count) + { + var ids = new List(count); + + for (var i = 0; i < count; i++) + ids.Add(i); + + return ids; + } + + private static string TextOf(McpToolResult result) => + result.Payload["content"]![0]!["text"]!.Value()!; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta new file mode 100644 index 00000000000..11c4db393c3 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/ListSceneEntitiesToolShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: fe60d9fd78ad61247aa09f22ac6ecab7 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs new file mode 100644 index 00000000000..d5e95c7581e --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs @@ -0,0 +1,50 @@ +using DCL.McpServer.Core; +using NUnit.Framework; +using System.Reflection; + +namespace DCL.McpServer.Tests +{ + /// + /// Covers McpHttpServer.IsAllowed, the DNS-rebinding guard that decides which Origin headers may + /// reach the JSON-RPC dispatcher. The method is private static and lives in Core, which these tests do + /// not modify, so it is exercised through reflection. If IsAllowed is later exposed as internal + /// (with InternalsVisibleTo), replace the reflection shim with a direct call. + /// + public class McpHttpServerShould + { + private static readonly MethodInfo IS_ALLOWED = + typeof(McpHttpServer).GetMethod("IsAllowed", BindingFlags.NonPublic | BindingFlags.Static)!; + + private static bool IsAllowed(string? origin) => + (bool)IS_ALLOWED.Invoke(null, new object?[] { origin })!; + + [TestCase(null)] + [TestCase("")] + public void AllowRequestsWithoutAnOrigin(string? origin) => + Assert.That(IsAllowed(origin), Is.True); + + [TestCase("http://localhost")] + [TestCase("http://localhost:8080")] + [TestCase("http://127.0.0.1")] + [TestCase("http://127.0.0.1:9001")] + [TestCase("https://127.0.0.1:9001")] + public void AllowLoopbackHttpOrigins(string origin) => + Assert.That(IsAllowed(origin), Is.True); + + [TestCase("http://evil.com")] + [TestCase("https://attacker.example:9001")] + [TestCase("http://127.0.0.1.evil.com")] // rebinding: a foreign host that merely starts with the loopback IP + public void RejectNonLoopbackHosts(string origin) => + Assert.That(IsAllowed(origin), Is.False); + + [TestCase("ftp://127.0.0.1")] + [TestCase("file:///etc/passwd")] + public void RejectNonHttpSchemes(string origin) => + Assert.That(IsAllowed(origin), Is.False); + + [TestCase("not-a-valid-origin")] + [TestCase("://missing-scheme")] + public void RejectMalformedOrigins(string origin) => + Assert.That(IsAllowed(origin), Is.False); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta new file mode 100644 index 00000000000..800c6157bd0 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpHttpServerShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5b82a6ebaa2cb0148808b9bc41a981d7 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs new file mode 100644 index 00000000000..b62b5eef522 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs @@ -0,0 +1,260 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Text.RegularExpressions; +using System.Threading; +using UnityEngine; +using UnityEngine.TestTools; + +namespace DCL.McpServer.Tests +{ + public class McpJsonRpcDispatcherShould + { + private const string SERVER_VERSION = "9.9.9-test"; + + [Test] + public void AnswerInitializeWithProtocolCapabilitiesAndServerInfo() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject result = ResultOf(Dispatch(dispatcher, Request(1, "initialize"))); + + Assert.That(result["protocolVersion"]!.Value(), Is.EqualTo("2025-06-18")); + Assert.That(result["protocolVersion"]!.Value(), Is.EqualTo(McpJsonRpcDispatcher.PROTOCOL_VERSION)); + + var capabilities = (JObject)result["capabilities"]!; + Assert.That(capabilities.ContainsKey("tools"), Is.True); + + var serverInfo = (JObject)result["serverInfo"]!; + Assert.That(serverInfo["version"]!.Value(), Is.EqualTo(SERVER_VERSION)); + Assert.That(serverInfo.ContainsKey("pid"), Is.True); + Assert.That(serverInfo["pid"]!.Type, Is.EqualTo(JTokenType.Integer)); + } + + [Test] + public void AnswerPingWithAnEmptyResult() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject result = ResultOf(Dispatch(dispatcher, Request(2, "ping"))); + + Assert.That(result.Count, Is.EqualTo(0)); + } + + [Test] + public void ListToolsWithInputSchemaAndAnnotations() + { + FakeMcpTool tool = FakeMcpTool.Returning("reader", annotations: McpToolAnnotations.ReadOnly()); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + JObject result = ResultOf(Dispatch(dispatcher, Request(3, "tools/list"))); + + var tools = (JArray)result["tools"]!; + Assert.That(tools.Count, Is.EqualTo(1)); + + JObject entry = ToolEntry(result, "reader"); + Assert.That(entry["description"]!.Value(), Is.EqualTo("reader description")); + + var inputSchema = (JObject)entry["inputSchema"]!; + Assert.That(inputSchema["type"]!.Value(), Is.EqualTo("object")); + Assert.That(inputSchema.ContainsKey("properties"), Is.True); + + var annotations = (JObject)entry["annotations"]!; + Assert.That(annotations["readOnlyHint"]!.Value(), Is.True); + } + + [Test] + public void CallAKnownToolAndWrapItsResult() + { + FakeMcpTool tool = FakeMcpTool.Returning("echo", McpToolResult.Text("done")); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + var arguments = new JObject { ["value"] = "hi" }; + JObject result = ResultOf(Dispatch(dispatcher, CallRequest(4, "echo", arguments))); + + Assert.That(tool.CallCount, Is.EqualTo(1)); + Assert.That(tool.LastArguments!["value"]!.Value(), Is.EqualTo("hi")); + + var content = (JArray)result["content"]!; + Assert.That(content[0]["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]["text"]!.Value(), Is.EqualTo("done")); + Assert.That(result.ContainsKey("isError"), Is.False); + } + + [Test] + public void CallWithEmptyArgumentsWhenNoneAreProvided() + { + FakeMcpTool tool = FakeMcpTool.Returning("echo"); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // A tools/call request carrying a name but no arguments object. + string request = new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = 5, + ["method"] = "tools/call", + ["params"] = new JObject { ["name"] = "echo" }, + }.ToString(Formatting.None); + + ResultOf(Dispatch(dispatcher, request)); + + Assert.That(tool.CallCount, Is.EqualTo(1)); + Assert.That(tool.LastArguments, Is.Not.Null); + Assert.That(tool.LastArguments!.Count, Is.EqualTo(0)); + } + + [Test] + public void RejectAnUnknownToolWithInvalidParams() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(FakeMcpTool.Returning("known")); + + JObject error = ErrorOf(Dispatch(dispatcher, CallRequest(6, "missing", new JObject()))); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32602)); + Assert.That(error["message"]!.Value(), Does.Contain("missing")); + } + + [Test] + public void ReportAToolFailureAsAnIsErrorResultNotAJsonRpcError() + { + FakeMcpTool tool = FakeMcpTool.Throwing("boom", new InvalidOperationException("kaboom")); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // ReportHub routes the caught exception to the Unity logger; expect it so the test does not fail. + LogAssert.Expect(LogType.Exception, new Regex("kaboom")); + + JObject response = JObject.Parse(Dispatch(dispatcher, CallRequest(7, "boom", new JObject()))!); + + // The failure is delivered inside result (isError), not as a top-level JSON-RPC error. + Assert.That(response.ContainsKey("error"), Is.False); + + var result = (JObject)response["result"]!; + Assert.That(result["isError"]!.Value(), Is.True); + Assert.That(result["content"]![0]!["text"]!.Value(), Does.Contain("boom")); + } + + [Test] + public void RethrowWhenAToolIsCancelled() + { + FakeMcpTool tool = FakeMcpTool.Throwing("cancelled", new OperationCanceledException()); + McpJsonRpcDispatcher dispatcher = DispatcherWith(tool); + + // Catch, not Throws: AsTask in the Dispatch helper surfaces the cancellation as the derived TaskCanceledException. + Assert.Catch(() => Dispatch(dispatcher, CallRequest(8, "cancelled", new JObject()))); + } + + [Test] + public void RejectAnUnknownMethodWithMethodNotFound() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject error = ErrorOf(Dispatch(dispatcher, Request(9, "resources/list"))); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32601)); + Assert.That(error["message"]!.Value(), Does.Contain("resources/list")); + } + + [Test] + public void DropNotificationsWithoutAnId() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + string request = new JObject + { + ["jsonrpc"] = "2.0", + ["method"] = "notifications/initialized", + }.ToString(Formatting.None); + + Assert.That(Dispatch(dispatcher, request), Is.Null); + } + + [Test] + public void ReplyWithAParseErrorOnMalformedJson() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + JObject error = ErrorOf(Dispatch(dispatcher, "{ this is not json")); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32700)); + } + + [Test] + public void RejectARequestMissingAMethodWithInvalidRequest() + { + McpJsonRpcDispatcher dispatcher = DispatcherWith(); + + string request = new JObject { ["jsonrpc"] = "2.0", ["id"] = 10 }.ToString(Formatting.None); + + JObject error = ErrorOf(Dispatch(dispatcher, request)); + + Assert.That(error["code"]!.Value(), Is.EqualTo(-32600)); + } + + private static McpJsonRpcDispatcher DispatcherWith(params McpTool[] tools) + { + var registry = new McpToolsRegistry(); + + foreach (McpTool tool in tools) + registry.Add(tool); + + registry.Build(); + return new McpJsonRpcDispatcher(registry, SERVER_VERSION); + } + + // AsTask allows a blocking wait: a tools/call response completes on the thread pool (the dispatcher + // hops there after the tool), so the UniTask is no longer synchronously completed on the test thread. + private static string? Dispatch(McpJsonRpcDispatcher dispatcher, string requestJson) => + dispatcher.DispatchAsync(requestJson, CancellationToken.None).AsTask().GetAwaiter().GetResult(); + + private static string Request(int id, string method) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["method"] = method, + }.ToString(Formatting.None); + + private static string CallRequest(int id, string toolName, JObject arguments) => + new JObject + { + ["jsonrpc"] = "2.0", + ["id"] = id, + ["method"] = "tools/call", + ["params"] = new JObject + { + ["name"] = toolName, + ["arguments"] = arguments, + }, + }.ToString(Formatting.None); + + private static JObject ResultOf(string? response) + { + Assert.That(response, Is.Not.Null, "expected a response, got a dropped message"); + var parsed = JObject.Parse(response!); + Assert.That(parsed["jsonrpc"]!.Value(), Is.EqualTo("2.0")); + Assert.That(parsed.ContainsKey("error"), Is.False, $"expected a result, got an error: {response}"); + return (JObject)parsed["result"]!; + } + + private static JObject ErrorOf(string? response) + { + Assert.That(response, Is.Not.Null, "expected an error response, got a dropped message"); + var parsed = JObject.Parse(response!); + Assert.That(parsed["jsonrpc"]!.Value(), Is.EqualTo("2.0")); + return (JObject)parsed["error"]!; + } + + private static JObject ToolEntry(JObject toolsListResult, string name) + { + foreach (JToken entry in (JArray)toolsListResult["tools"]!) + if (entry["name"]!.Value() == name) + return (JObject)entry; + + Assert.Fail($"tool '{name}' not found in tools/list"); + return null!; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta new file mode 100644 index 00000000000..cd4bb0fe2a8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonRpcDispatcherShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e447afff3fddc6746922385663f764be \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs new file mode 100644 index 00000000000..9e3aed6e603 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs @@ -0,0 +1,119 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System.Diagnostics.CodeAnalysis; + +namespace DCL.McpServer.Tests +{ + public class McpJsonSchemaShould + { + [SuppressMessage("ReSharper", "UnusedMember.Local")] // the members are read through reflection by McpWireEnum + private enum Speed + { + WALK, + RUN, + } + + [Test] + public void EmitTheDeclaredJsonSchemaTypeForEachFieldKind() + { + JObject schema = McpJsonSchema.Object() + .Number("ratio") + .Boolean("flag") + .Build(); + + var properties = (JObject)schema["properties"]!; + Assert.That(properties["ratio"]!["type"]!.Value(), Is.EqualTo("number")); + Assert.That(properties["flag"]!["type"]!.Value(), Is.EqualTo("boolean")); + } + + [Test] + public void ConstrainAnEnumFieldToTheWireNamesOfItsMembers() + { + JObject schema = McpJsonSchema.Object() + .Enum("kind") + .Enum("mode", allowed: new[] { Speed.RUN }) + .Build(); + + var properties = (JObject)schema["properties"]!; + Assert.That(((JArray)properties["kind"]!["enum"]!).ToObject(), Is.EqualTo(new[] { "walk", "run" })); + Assert.That(((JArray)properties["mode"]!["enum"]!).ToObject(), Is.EqualTo(new[] { "run" })); + } + + [Test] + public void OmitDescriptionAndEnumWhenNotProvided() + { + JObject schema = McpJsonSchema.Object().String("name").Build(); + + var field = (JObject)schema["properties"]!["name"]!; + Assert.That(field.ContainsKey("description"), Is.False); + Assert.That(field.ContainsKey("enum"), Is.False); + } + + [Test] + public void CollectEveryRequiredFieldInDeclarationOrder() + { + JObject schema = McpJsonSchema.Object() + .String("first", isRequired: true) + .Integer("skipped") + .Boolean("second", isRequired: true) + .Build(); + + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "first", "second" })); + } + + [Test] + public void NestAnObjectFieldWithItsOwnProperties() + { + JObject schema = McpJsonSchema.Object() + .Object("camera", McpJsonSchema.Object().String("mode"), "The camera.", isRequired: true) + .Build(); + + var camera = (JObject)schema["properties"]!["camera"]!; + Assert.That(camera["type"]!.Value(), Is.EqualTo("object")); + Assert.That(camera["description"]!.Value(), Is.EqualTo("The camera.")); + Assert.That(camera["properties"]!["mode"]!["type"]!.Value(), Is.EqualTo("string")); + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "camera" })); + } + + [Test] + public void AdmitNullAlongsideAnObjectForANullableNestedField() + { + JObject schema = McpJsonSchema.Object() + .Object("scene", McpJsonSchema.Object().String("name"), nullable: true) + .Build(); + + var sceneType = (JArray)schema["properties"]!["scene"]!["type"]!; + Assert.That(sceneType.ToObject(), Is.EqualTo(new[] { "object", "null" })); + } + + [Test] + public void AdmitNullAlongsideTheDeclaredTypeForANullableScalar() + { + JObject schema = McpJsonSchema.Object().String("address", nullable: true).Build(); + + var addressType = (JArray)schema["properties"]!["address"]!["type"]!; + Assert.That(addressType.ToObject(), Is.EqualTo(new[] { "string", "null" })); + } + + [Test] + public void DescribeAnArrayOfIntegerItems() + { + JObject schema = McpJsonSchema.Object().IntegerArray("entityIds").Build(); + + var field = (JObject)schema["properties"]!["entityIds"]!; + Assert.That(field["type"]!.Value(), Is.EqualTo("array")); + Assert.That(field["items"]!["type"]!.Value(), Is.EqualTo("integer")); + } + + [Test] + public void ProduceAnEmptyPropertiesObjectForAnArgumentlessTool() + { + JObject schema = McpJsonSchema.Object().Build(); + + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + Assert.That(((JObject)schema["properties"]!).Count, Is.EqualTo(0)); + Assert.That(schema.ContainsKey("required"), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta new file mode 100644 index 00000000000..0b62cbacd67 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpJsonSchemaShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: bcaebf033b5a3154b84bfdffe0aa383e \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpPointerEventSystemShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpPointerEventSystemShould.cs new file mode 100644 index 00000000000..eadd22179af --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpPointerEventSystemShould.cs @@ -0,0 +1,386 @@ +using Arch.Core; +using CRDT; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.ECSComponents; +using DCL.Interaction.Utility; +using DCL.McpServer.Components; +using DCL.McpServer.Systems; +using DCL.Utilities; +using ECS.SceneLifeCycle; +using ECS.TestSuite; +using ECS.Unity.Transforms.Components; +using NSubstitute; +using NUnit.Framework; +using SceneRunner.Scene; +using UnityEngine; +using Utility.Multithreading; + +namespace DCL.McpServer.Tests +{ + public class McpPointerEventSystemShould : UnitySystemTestBase + { + private const int TARGET_CRDT_ID = 512; + private const int BLOCKER_CRDT_ID = 513; + + private World sceneWorld = null!; + private Entity playerEntity; + private Entity targetEntity; + + private GameObject cameraGo = null!; + private GameObject playerGo = null!; + private GameObject targetGo = null!; + private GameObject blockerGo = null!; + + private BoxCollider targetCollider = null!; + private PBPointerEvents targetPointerEvents = null!; + private ISceneStateProvider sceneStateProvider = null!; + private uint tick; + + [SetUp] + public void SetUp() + { + sceneWorld = World.Create(); + + cameraGo = new GameObject("mcp-click-test-camera"); + Camera camera = cameraGo.AddComponent(); + world.Create(new CameraComponent(camera)); // Mode defaults to FirstPerson + + playerGo = new GameObject("mcp-click-test-player"); + playerEntity = world.Create(new CharacterTransform(playerGo.transform)); + + targetGo = new GameObject("mcp-click-test-target") + { + transform = { position = new Vector3(0f, 0f, 5f), }, + }; + + targetCollider = targetGo.AddComponent(); + + targetPointerEvents = new PBPointerEvents + { + PointerEvents = + { + new PBPointerEvents.Types.Entry + { + EventType = PointerEventType.PetDown, + EventInfo = new PBPointerEvents.Types.Info + { + Button = InputAction.IaPointer, + HoverText = "Open", + MaxDistance = 10f, + }, + }, + }, + }; + + targetPointerEvents.AppendPointerEventResultsIntent.InitializeWithAlloc(); + + // The entity needs a TransformComponent so ResolveEntityAimPoint can aim the validation ray at it; + // without it the aim point is Vector3.zero and the delivery bails out before the raycast. + targetEntity = sceneWorld.Create(targetPointerEvents, new CRDTEntity(TARGET_CRDT_ID), new TransformComponent(targetGo.transform)); + + // Colliders created/moved this frame are not in the PhysX scene until transforms are synced (no physics step runs in EditMode). + Physics.SyncTransforms(); + + tick = 100u; + sceneStateProvider = Substitute.For(); + sceneStateProvider.IsCurrent.Returns(true); + sceneStateProvider.State.Returns(new Atomic(SceneState.Running)); + sceneStateProvider.TickNumber.Returns(_ => tick); + + ISceneFacade sceneFacade = Substitute.For(); + sceneFacade.SceneStateProvider.Returns(sceneStateProvider); + sceneFacade.EcsExecutor.Returns(new SceneEcsExecutor(sceneWorld)); + + IReadonlyReactiveProperty currentScene = Substitute.For>(); + currentScene.Value.Returns(sceneFacade); + + IScenesCache scenesCache = Substitute.For(); + scenesCache.CurrentScene.Returns(currentScene); + + IEntityCollidersGlobalCache collidersCache = Substitute.For(); + + collidersCache.TryGetSceneEntity(Arg.Any(), out Arg.Any()) + .Returns(call => + { + var collider = call.ArgAt(0); + + if (collider == targetCollider) + { + call[1] = new GlobalColliderSceneEntityInfo( + new SceneEcsExecutor(sceneWorld), + new ColliderSceneEntityInfo(targetEntity, new CRDTEntity(TARGET_CRDT_ID), ColliderLayer.ClPointer)); + + return true; + } + + if (blockerGo != null && collider == blockerGo.GetComponent()) + { + call[1] = new GlobalColliderSceneEntityInfo( + new SceneEcsExecutor(sceneWorld), + new ColliderSceneEntityInfo(sceneWorld.Create(new CRDTEntity(BLOCKER_CRDT_ID)), new CRDTEntity(BLOCKER_CRDT_ID), ColliderLayer.ClPhysics)); + + return true; + } + + return false; + }); + + system = new McpPointerEventSystem(world, scenesCache, collidersCache, playerEntity); + system.Initialize(); + } + + protected override void OnTearDown() + { + Object.DestroyImmediate(cameraGo); + Object.DestroyImmediate(playerGo); + Object.DestroyImmediate(targetGo); + + if (blockerGo != null) + Object.DestroyImmediate(blockerGo); + + sceneWorld.Dispose(); + } + + private UniTaskCompletionSource AddIntent( + PointerEventType eventType = PointerEventType.PetDown, + int? targetId = null, + McpPressHandoff? press = null) + { + var completion = new UniTaskCompletionSource(); + + world.Add(playerEntity, new McpEcsPointerEventIntent(targetId ?? targetEntity.Id, null, InputAction.IaPointer, eventType, press) + { + Completion = completion, + }); + + return completion; + } + + /// Delivers a press and returns its result, asserting the handoff the release leg needs is filled. + private McpPointerClickResult DeliverPress() + { + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.True); + Assert.That(result.Press, Is.Not.Null); + + // The scene-world flush clears the intent at the end of a real frame. + targetPointerEvents.AppendPointerEventResultsIntent.Clear(); + + return result; + } + + private static McpPointerClickResult ResultOf(UniTaskCompletionSource completion) + { + Assert.That(completion.Task.Status, Is.EqualTo(UniTaskStatus.Succeeded)); + return completion.Task.GetAwaiter().GetResult(); + } + + [Test] + public void DeliverPressThenOrderedReleaseOnNextTick() + { + UniTaskCompletionSource pressCompletion = AddIntent(); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetDown))); + + McpPointerClickResult pressResult = ResultOf(pressCompletion); + Assert.That(pressResult.Hit, Is.True); + Assert.That(pressResult.CrdtEntityId, Is.EqualTo(TARGET_CRDT_ID)); + Assert.That(pressResult.HoverText, Is.EqualTo("Open")); + Assert.That(pressResult.Press, Is.Not.Null); + Assert.That(pressResult.Press.Value.Entity, Is.EqualTo(targetEntity)); + Assert.That(pressResult.Press.Value.Tick, Is.EqualTo(tick)); + Assert.That(world.Has(playerEntity), Is.False); + + // The scene-world flush clears the intent at the end of a real frame. + targetPointerEvents.AppendPointerEventResultsIntent.Clear(); + + UniTaskCompletionSource releaseCompletion = AddIntent(PointerEventType.PetUp, press: pressResult.Press); + + system.Update(0); // same tick: keeps waiting so PetUp lands on a later tick than PetDown + Assert.That(releaseCompletion.Task.Status, Is.EqualTo(UniTaskStatus.Pending)); + + tick++; + system.Update(0); + + actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetUp))); + + McpPointerClickResult releaseResult = ResultOf(releaseCompletion); + Assert.That(releaseResult.Hit, Is.True); + Assert.That(releaseResult.UpRayMissed, Is.False); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void DeliverSinglePressWithoutWaiting() + { + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetDown))); + + Assert.That(ResultOf(completion).Hit, Is.True); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void DeliverSingleReleaseWithoutPressContext() + { + UniTaskCompletionSource completion = AddIntent(PointerEventType.PetUp); + + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetUp))); + + Assert.That(ResultOf(completion).Hit, Is.True); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void ReplayPressHitWhenReleaseRayIsBlocked() + { + McpPointerClickResult pressResult = DeliverPress(); + + // A blocker slides between the camera and the target after the press. + blockerGo = new GameObject("mcp-click-test-blocker") { transform = { position = new Vector3(0f, 0f, 2f) }}; + + blockerGo.AddComponent(); + Physics.SyncTransforms(); + + UniTaskCompletionSource releaseCompletion = AddIntent(PointerEventType.PetUp, press: pressResult.Press); + + tick++; + system!.Update(0); + + var actions = targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions; + Assert.That(actions.Count, Is.EqualTo(1)); + Assert.That(actions[0], Is.EqualTo((InputAction.IaPointer, PointerEventType.PetUp))); + + McpPointerClickResult releaseResult = ResultOf(releaseCompletion); + Assert.That(releaseResult.Hit, Is.True); + Assert.That(releaseResult.UpRayMissed, Is.True); + } + + [Test] + public void ReportPressOnlyWhenTargetDiesBeforeRelease() + { + McpPointerClickResult pressResult = DeliverPress(); + + sceneWorld.Destroy(targetEntity); + + UniTaskCompletionSource releaseCompletion = AddIntent(PointerEventType.PetUp, press: pressResult.Press); + + tick++; + system!.Update(0); + + McpPointerClickResult releaseResult = ResultOf(releaseCompletion); + Assert.That(releaseResult.Hit, Is.False); + Assert.That(releaseResult.UpRayMissed, Is.True); + Assert.That(releaseResult.FailureReason, Does.Contain("destroyed")); + } + + [Test] + public void FailReleaseWhenSceneWorldChangedMidClick() + { + McpPointerClickResult pressResult = DeliverPress(); + + var reloadedWorld = World.Create(); + + try + { + McpPressHandoff stalePress = pressResult.Press.Value; + stalePress.World = reloadedWorld; + + UniTaskCompletionSource releaseCompletion = AddIntent(PointerEventType.PetUp, press: stalePress); + + tick++; + system!.Update(0); + + McpPointerClickResult releaseResult = ResultOf(releaseCompletion); + Assert.That(releaseResult.Hit, Is.False); + Assert.That(releaseResult.FailureReason, Does.Contain("reloaded")); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + } + finally + { + reloadedWorld.Dispose(); + } + } + + [Test] + public void FailWhenAnotherColliderBlocksTheRay() + { + blockerGo = new GameObject("mcp-click-test-blocker") { transform = { position = new Vector3(0f, 0f, 2f) }}; + + blockerGo.AddComponent(); + Physics.SyncTransforms(); + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.BlockedByCrdtId, Is.EqualTo(BLOCKER_CRDT_ID)); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + Assert.That(world.Has(playerEntity), Is.False); + } + + [Test] + public void FailWhenOutOfRange() + { + targetPointerEvents.PointerEvents[0].EventInfo.MaxDistance = 2f; + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("out of range")); + Assert.That(targetPointerEvents.AppendPointerEventResultsIntent.ValidInputActions.Count, Is.EqualTo(0)); + } + + [Test] + public void FailWhenEntityHasNoPointerEvents() + { + sceneWorld.Remove(targetEntity); + + UniTaskCompletionSource completion = AddIntent(); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("PointerEvents")); + } + + [Test] + public void FailWhenEntityIdIsUnknown() + { + UniTaskCompletionSource completion = AddIntent(targetId: 987654); + + system!.Update(0); + + McpPointerClickResult result = ResultOf(completion); + Assert.That(result.Hit, Is.False); + Assert.That(result.FailureReason, Does.Contain("no entity")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpPointerEventSystemShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpPointerEventSystemShould.cs.meta new file mode 100644 index 00000000000..31ec606ea26 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpPointerEventSystemShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 3245eba0487e4b8a8c8f125b10268888 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpRequestShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpRequestShould.cs new file mode 100644 index 00000000000..17f393db1de --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpRequestShould.cs @@ -0,0 +1,78 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using NUnit.Framework; + +namespace DCL.McpServer.Tests +{ + public class McpRequestShould + { + private struct TestEcsRequest : IMcpEcsRequest + { + public int Payload; + public UniTaskCompletionSource? Completion { get; set; } + } + + private World world = null!; + private Entity entity; + + [SetUp] + public void SetUp() + { + world = World.Create(); + entity = world.Create(); + } + + [TearDown] + public void TearDown() + { + world.Dispose(); + } + + [Test] + public void InstallRequestWithPendingCompletion() + { + UniTask task = McpRequest.SendAsync(world, entity, new TestEcsRequest { Payload = 7 }, -1); + + Assert.That(task.Status, Is.EqualTo(UniTaskStatus.Pending)); + Assert.That(world.TryGet(entity, out TestEcsRequest installed), Is.True); + Assert.That(installed.Payload, Is.EqualTo(7)); + Assert.That(installed.Completion, Is.Not.Null); + } + + [Test] + public void PreemptPendingRequestWhenNewerOneIsSent() + { + UniTask first = McpRequest.SendAsync(world, entity, new TestEcsRequest { Payload = 1 }, -1); + UniTask second = McpRequest.SendAsync(world, entity, new TestEcsRequest { Payload = 2 }, -1); + + Assert.That(first.Status, Is.EqualTo(UniTaskStatus.Succeeded)); + Assert.That(first.GetAwaiter().GetResult(), Is.EqualTo(-1)); + Assert.That(second.Status, Is.EqualTo(UniTaskStatus.Pending)); + Assert.That(world.Get(entity).Payload, Is.EqualTo(2)); + } + + [Test] + public void CompleteAndRemoveResolvesAwaiterAfterRemoval() + { + UniTask task = McpRequest.SendAsync(world, entity, new TestEcsRequest(), -1); + TestEcsRequest ecsRequest = world.Get(entity); + + McpRequest.CompleteAndRemove(world, entity, ecsRequest, 42); + + Assert.That(world.Has(entity), Is.False); + Assert.That(task.Status, Is.EqualTo(UniTaskStatus.Succeeded)); + Assert.That(task.GetAwaiter().GetResult(), Is.EqualTo(42)); + } + + [Test] + public void CompleteAndRemoveTolerateMissingCompletion() + { + world.Add(entity, new TestEcsRequest()); + + McpRequest.CompleteAndRemove(world, entity, world.Get(entity), 0); + + Assert.That(world.Has(entity), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpRequestShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpRequestShould.cs.meta new file mode 100644 index 00000000000..04eb44e2a74 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpRequestShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 609353fd62c82d64787cc455fd815f74 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs new file mode 100644 index 00000000000..851ebca12fc --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs @@ -0,0 +1,57 @@ +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System.Collections.Generic; + +namespace DCL.McpServer.Tests +{ + /// + /// Guards against output-schema ↔ structuredContent drift: a tool declares its OutputSchema by hand and + /// builds the matching payload by hand, so a field added to one and forgotten in the other lets the schema + /// silently misdescribe the payload to the agent. Asserting the two carry the same property names + /// (recursively, into nested objects the payload populates) turns that drift into a failing test. + /// + internal static class McpSchemaAssert + { + public static void KeysMatch(JObject schema, JObject payload) + { + var properties = schema["properties"] as JObject ?? new JObject(); + + CollectionAssert.AreEquivalent(NamesOf(properties), NamesOf(payload), + $"Output schema and payload disagree on the keys of '{PathOf(schema)}'."); + + foreach (JProperty property in properties.Properties()) + if (DeclaresObject(property.Value) && payload[property.Name] is JObject nested) + KeysMatch((JObject)property.Value, nested); + } + + private static List NamesOf(JObject obj) + { + var names = new List(); + + foreach (JProperty property in obj.Properties()) + names.Add(property.Name); + + return names; + } + + // A property models an object both as type "object" and as a nullable ["object", "null"] union. + private static bool DeclaresObject(JToken schema) + { + JToken? type = schema["type"]; + + if (type is JArray union) + { + foreach (JToken member in union) + if (member.Value() == "object") + return true; + + return false; + } + + return type?.Value() == "object"; + } + + private static string PathOf(JObject schema) => + string.IsNullOrEmpty(schema.Path) ? "" : schema.Path; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta new file mode 100644 index 00000000000..a88c5ac22bd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpSchemaAssert.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: b0e1283f51dcb2948b8bb95431335e2f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs new file mode 100644 index 00000000000..e99d815e797 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs @@ -0,0 +1,78 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Text; + +namespace DCL.McpServer.Tests +{ + public class McpToolResultShould + { + [Test] + public void WrapTextInASingleTextContentItemWithoutAnErrorFlag() + { + JObject payload = McpToolResult.Text("hello").Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(1)); + Assert.That(content[0]["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]["text"]!.Value(), Is.EqualTo("hello")); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + + [Test] + public void CarryBothAMirroringTextItemAndStructuredContent() + { + var structured = new JObject + { + ["count"] = 3, + ["nested"] = new JObject { ["ok"] = true }, + }; + + JObject payload = McpToolResult.TextWithStructured("mirror", structured).Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(1)); + Assert.That(content[0]["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]["text"]!.Value(), Is.EqualTo("mirror")); + + var structuredContent = (JObject)payload["structuredContent"]!; + Assert.That(structuredContent["count"]!.Value(), Is.EqualTo(3)); + Assert.That(structuredContent["nested"]!["ok"]!.Value(), Is.True); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + + [Test] + public void FlagErrorsWithIsErrorAndCarryTheMessageAsText() + { + JObject payload = McpToolResult.Error("it broke").Payload; + + Assert.That(payload["isError"]!.Value(), Is.True); + + var content = (JArray)payload["content"]!; + Assert.That(content[0]["type"]!.Value(), Is.EqualTo("text")); + Assert.That(content[0]["text"]!.Value(), Is.EqualTo("it broke")); + } + + [Test] + public void EncodeImageBytesAsBase64AlongsideACaption() + { + byte[] bytes = Encoding.UTF8.GetBytes("pixels"); + + JObject payload = McpToolResult.Image(bytes, "image/png", "a screenshot").Payload; + + var content = (JArray)payload["content"]!; + Assert.That(content.Count, Is.EqualTo(2)); + + var image = (JObject)content[0]; + Assert.That(image["type"]!.Value(), Is.EqualTo("image")); + Assert.That(image["mimeType"]!.Value(), Is.EqualTo("image/png")); + Assert.That(Convert.FromBase64String(image["data"]!.Value()!), Is.EqualTo(bytes)); + + var caption = (JObject)content[1]; + Assert.That(caption["type"]!.Value(), Is.EqualTo("text")); + Assert.That(caption["text"]!.Value(), Is.EqualTo("a screenshot")); + Assert.That(payload.ContainsKey("isError"), Is.False); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta new file mode 100644 index 00000000000..85c2acbac61 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolResultShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 746fe351b002c704ea5996a66566492f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs new file mode 100644 index 00000000000..0359de2f052 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs @@ -0,0 +1,208 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using NUnit.Framework; +using System; +using System.Collections.Generic; +using System.Threading; + +namespace DCL.McpServer.Tests +{ + public class McpToolsRegistryShould + { + [Test] + public void BuildAnObjectSchemaWithTypedPropertiesAndRequired() + { + // Act + JObject schema = McpJsonSchema.Object() + .Integer("count", "How many.") + .String("mode", "Pick one.", enumValues: new[] { "a", "b" }, isRequired: true) + .Build(); + + // Assert + Assert.That(schema["type"]!.Value(), Is.EqualTo("object")); + + var properties = (JObject)schema["properties"]!; + Assert.That(properties["count"]!["type"]!.Value(), Is.EqualTo("integer")); + Assert.That(properties["count"]!["description"]!.Value(), Is.EqualTo("How many.")); + Assert.That(properties["mode"]!["type"]!.Value(), Is.EqualTo("string")); + Assert.That(((JArray)properties["mode"]!["enum"]!).ToObject(), Is.EqualTo(new[] { "a", "b" })); + + Assert.That(((JArray)schema["required"]!).ToObject(), Is.EqualTo(new[] { "mode" })); + } + + [Test] + public void OmitRequiredWhenNoFieldIsRequired() + { + // Act + JObject schema = McpJsonSchema.Object().Boolean("flag").Build(); + + // Assert + Assert.That(schema.ContainsKey("required"), Is.False); + } + + [Test] + public void FailRegistrationNamingTheToolWithAnInvalidOutputSchema() + { + // Arrange + var registry = new McpToolsRegistry() + .Add(new FakeTool("broken", McpToolAnnotations.ReadOnly(), outputSchema: new JObject())); + + // Act & Assert + InvalidOperationException error = Assert.Throws(() => registry.Build()); + Assert.That(error.Message, Does.Contain("broken")); + } + + [Test] + public void EmitReadOnlyAnnotationsWithoutStateChangeHints() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("reader", McpToolAnnotations.ReadOnly())) + .Build()); + + // Act + JObject annotations = AnnotationsOf(toolsList, "reader"); + + // Assert + Assert.That(annotations["readOnlyHint"]!.Value(), Is.True); + Assert.That(annotations["openWorldHint"]!.Value(), Is.False); + Assert.That(annotations.ContainsKey("destructiveHint"), Is.False); + Assert.That(annotations.ContainsKey("idempotentHint"), Is.False); + } + + [Test] + public void EmitMutatingAnnotationsWithAllStateChangeHints() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("mutator", McpToolAnnotations.Mutating(destructive: true, idempotent: false))) + .Build()); + + // Act + JObject annotations = AnnotationsOf(toolsList, "mutator"); + + // Assert + Assert.That(annotations["readOnlyHint"]!.Value(), Is.False); + Assert.That(annotations["destructiveHint"]!.Value(), Is.True); + Assert.That(annotations["idempotentHint"]!.Value(), Is.False); + Assert.That(annotations["openWorldHint"]!.Value(), Is.False); + } + + [Test] + public void IncludeOutputSchemaWhenTheToolDeclaresOne() + { + // Arrange + JObject outputSchema = McpJsonSchema.Object().Integer("total").Build(); + + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("structured", McpToolAnnotations.ReadOnly(), outputSchema: outputSchema)) + .Build()); + + // Act + JObject entry = EntryOf(toolsList, "structured"); + + // Assert + Assert.That(entry.ContainsKey("outputSchema"), Is.True); + Assert.That(entry["outputSchema"]!["type"]!.Value(), Is.EqualTo("object")); + } + + [Test] + public void OmitOutputSchemaWhenTheToolDeclaresNone() + { + // Arrange + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("plain", McpToolAnnotations.ReadOnly())) + .Build()); + + // Act & Assert + Assert.That(EntryOf(toolsList, "plain").ContainsKey("outputSchema"), Is.False); + } + + [Test] + public void FindARegisteredToolByName() + { + var tool = new FakeTool("known", McpToolAnnotations.ReadOnly()); + var registry = new McpToolsRegistry().Add(tool); + + bool found = registry.TryGet("known", out McpTool? resolved); + + Assert.That(found, Is.True); + Assert.That(resolved, Is.SameAs(tool)); + } + + [Test] + public void NotFindAnUnknownTool() + { + var registry = new McpToolsRegistry().Add(new FakeTool("known", McpToolAnnotations.ReadOnly())); + + Assert.That(registry.TryGet("missing", out McpTool? resolved), Is.False); + Assert.That(resolved, Is.Null); + } + + [Test] + public void NotFindAToolForANullOrEmptyName() + { + var registry = new McpToolsRegistry().Add(new FakeTool("known", McpToolAnnotations.ReadOnly())); + + Assert.That(registry.TryGet(null, out McpTool? byNull), Is.False); + Assert.That(byNull, Is.Null); + + Assert.That(registry.TryGet(string.Empty, out McpTool? byEmpty), Is.False); + Assert.That(byEmpty, Is.Null); + } + + [Test] + public void ReflectTheRegisteredSetInTheToolsList() + { + JObject toolsList = Payload(new McpToolsRegistry() + .Add(new FakeTool("first", McpToolAnnotations.ReadOnly())) + .Add(new FakeTool("second", McpToolAnnotations.Mutating(destructive: false, idempotent: true))) + .Build()); + + var names = new List(); + + foreach (JToken entry in (JArray)toolsList["tools"]!) + names.Add(entry["name"]!.Value()!); + + Assert.That(names, Is.EquivalentTo(new[] { "first", "second" })); + } + + private static JObject Payload(McpToolsRegistry registry) => + JObject.Parse(registry.ToolsListPayload().ToString()); + + private static JObject AnnotationsOf(JObject toolsList, string name) => + (JObject)EntryOf(toolsList, name)["annotations"]!; + + private static JObject EntryOf(JObject toolsList, string name) + { + foreach (JToken entry in (JArray)toolsList["tools"]!) + if (entry["name"]!.Value() == name) + return (JObject)entry; + + Assert.Fail($"tool '{name}' not found in tools/list"); + return null!; + } + + private class FakeTool : McpTool + { + public override string Name { get; } + + public override McpToolAnnotations Annotations { get; } + + public override string Description => "fake"; + + public override JObject? OutputSchema { get; } + + public FakeTool(string name, McpToolAnnotations annotations, JObject? outputSchema = null) + { + Name = name; + Annotations = annotations; + OutputSchema = outputSchema; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) => + UniTask.FromResult(McpToolResult.Text("fake")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta new file mode 100644 index 00000000000..94d6bf30834 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpToolsRegistryShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1e7c91136f9767a4b9b3922fef87f114 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpWireEnumShould.cs b/Explorer/Assets/DCL/McpServer/Tests/McpWireEnumShould.cs new file mode 100644 index 00000000000..272a0c98e0b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpWireEnumShould.cs @@ -0,0 +1,57 @@ +using DCL.McpServer.Core; +using NUnit.Framework; +using System.Diagnostics.CodeAnalysis; + +namespace DCL.McpServer.Tests +{ + public class McpWireEnumShould + { + /// + /// Deliberately mixes every member-naming style the snake_case conversion must handle — PascalCase, + /// an acronym run, SCREAMING and SCREAMING_SNAKE — mirroring the real wire enums (CameraMode.FirstPerson, + /// CameraMode.SDKCamera, MovementKind.WALK). The members are read only through + /// reflection by McpWireEnum, hence the suppressions. + /// + [SuppressMessage("ReSharper", "InconsistentNaming")] + [SuppressMessage("ReSharper", "UnusedMember.Local")] + private enum Casing + { + FirstPerson, + WALK, + DroneView, + SDKCamera, + WAIT_TICK, + } + + [Test] + public void DeriveSnakeCaseWireNamesFromMemberNames() + { + Assert.That(McpWireEnum.WIRE_NAMES, Is.EqualTo(new[] { "first_person", "walk", "drone_view", "sdk_camera", "wait_tick" })); + } + + [Test] + public void ParseAWireNameBackToItsMember() + { + Assert.That(McpWireEnum.TryParse("drone_view", out Casing value), Is.True); + Assert.That(value, Is.EqualTo(Casing.DroneView)); + } + + [Test] + public void RejectAMemberNameThatIsNotInWireForm() + { + Assert.That(McpWireEnum.TryParse("DroneView", out _), Is.False); + } + + [Test] + public void FormatAMemberAsItsWireName() + { + Assert.That(McpWireEnum.ToWire(Casing.SDKCamera), Is.EqualTo("sdk_camera")); + } + + [Test] + public void NarrowWireNamesToASubsetOfMembers() + { + Assert.That(McpWireEnum.WireNamesOf(new[] { Casing.WALK, Casing.FirstPerson }), Is.EqualTo(new[] { "walk", "first_person" })); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tests/McpWireEnumShould.cs.meta b/Explorer/Assets/DCL/McpServer/Tests/McpWireEnumShould.cs.meta new file mode 100644 index 00000000000..b56a822f44b --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tests/McpWireEnumShould.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 382a79b03fe1172429962419729018ff \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools.meta b/Explorer/Assets/DCL/McpServer/Tools.meta new file mode 100644 index 00000000000..ea160a5cffd --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 9a8b8d76e6c14409697c08478f574549 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs new file mode 100644 index 00000000000..b24790d0278 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs @@ -0,0 +1,183 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Presses a pointer button on a scene entity by composing single-event s + /// delivered by McpPointerEventSystem, which validates the aim with the same raycast rules as the reticle + /// pipeline before filling the entity's pointer-event intent like a real click. A full click is a press + /// followed by a release ordered onto a later scene tick, merged into one result here. + /// + public class ClickEntityTool : McpTool + { + /// Wire-facing subset of : only the three pointer buttons make sense for a click. + private enum PointerButton : byte + { + POINTER, + PRIMARY, + SECONDARY, + } + + /// Wire-facing gesture kinds: a full click, or a single press/release leg. + private enum ClickKind : byte + { + /// Pointer down, then pointer up on the next scene tick. + CLICK, + DOWN, + UP, + } + + private const float DEFAULT_TIMEOUT_SEC = 3f; + private const float MIN_TIMEOUT_SEC = 0.5f; + private const float MAX_TIMEOUT_SEC = 15f; + + private readonly World world; + private readonly Entity playerEntity; + + public override string Name => "click_entity"; + + public override string Description => + "Press and release a pointer button on a scene entity so its PointerEvents fire exactly like a real click. " + + "The aim is validated by a physics raycast from the camera: occluders and the entity's maxDistance apply, and a miss " + + "returns hit:false with the blocking entity. Ids come from list_scene_entities. For entities whose collider " + + "sits away from their pivot (e.g. GLTF meshes), pass an explicit x/y/z world point to aim at."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Integer("entityId", "Target entity id in the current scene world (from list_scene_entities). Omit only when x/y/z are given, then the ray decides the target.") + .Number("x", "World-space aim point; overrides the automatic aim at the entity's collider center.") + .Number("y") + .Number("z") + .Enum("button", "Which input action to press. Default pointer (left click / IA_POINTER).") + .Enum("eventType", "click = down, then up on the next scene tick. Default click.") + .Number("timeoutSec", "Seconds to wait for delivery. Default 3, max 15."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public ClickEntityTool(World world, Entity playerEntity) + { + this.world = world; + this.playerEntity = playerEntity; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + bool hasEntityId = arguments.TryGetInt("entityId", out int entityId); + + bool hasAimPoint = arguments.TryGetFloat("x", out float x) + & arguments.TryGetFloat("y", out float y) + & arguments.TryGetFloat("z", out float z); + + if (!hasEntityId && !hasAimPoint) + return McpToolResult.Error("Provide entityId, or a full x/y/z world aim point, or both."); + + if (!arguments.TryGetEnum("button", PointerButton.POINTER, out PointerButton pointerButton)) + return McpToolResult.Error("button must be one of: pointer, primary, secondary."); + + InputAction button = pointerButton switch + { + PointerButton.PRIMARY => InputAction.IaPrimary, + PointerButton.SECONDARY => InputAction.IaSecondary, + _ => InputAction.IaPointer, + }; + + if (!arguments.TryGetEnum("eventType", ClickKind.CLICK, out ClickKind kind)) + return McpToolResult.Error("eventType must be one of: click, down, up."); + + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + int targetEntityId = hasEntityId ? entityId : -1; + Vector3? aimPoint = hasAimPoint ? new Vector3(x, y, z) : null; + + McpPointerClickResult result; + + try + { + // A single budget for the whole gesture: it covers both a paused simulation that never runs + // the system and a release stuck waiting for the scene tick to advance. + result = await RunGestureAsync(targetEntityId, aimPoint, button, kind) + .AttachExternalCancellation(ct) + .Timeout(TimeSpan.FromSeconds(timeoutSec)); + } + catch (TimeoutException) + { + await McpRequest.AbandonAsync(world, playerEntity); + return McpToolResult.Error($"click_entity did not complete within {timeoutSec}s (is the simulation paused?)."); + } + + var json = new JObject + { + ["hit"] = result.Hit, + ["entityId"] = result.SceneEntityId, + ["crdtEntityId"] = result.CrdtEntityId, + }; + + if (result.FailureReason != null) + json["reason"] = result.FailureReason; + + if (result.Hit) + { + json["hitPoint"] = result.HitPoint.ToVector(); + json["distance"] = Math.Round(result.Distance, 2); + } + + if (result.HoverText != null) + json["hoverText"] = result.HoverText; + + if (result.BlockedByEntityId != null) + { + json["blockedByEntityId"] = result.BlockedByEntityId; + json["blockedByCrdtId"] = result.BlockedByCrdtId; + json["blockedByCollider"] = result.BlockedByColliderName; + } + + if (result.UpRayMissed) + json["upRayMissed"] = true; + + return McpToolResult.Json(json); + } + + /// + /// Composes the requested gesture from single-event intents: a lone press or release is one delivery; + /// a click is a press followed by a release that carries the press handoff so the system keeps it + /// ordered onto a later scene tick. + /// + private async UniTask RunGestureAsync(int targetEntityId, Vector3? aimPoint, InputAction button, ClickKind kind) + { + PointerEventType pressType = kind == ClickKind.UP ? PointerEventType.PetUp : PointerEventType.PetDown; + + McpPointerClickResult down = await SendAsync(new McpEcsPointerEventIntent(targetEntityId, aimPoint, button, pressType)); + + if (kind != ClickKind.CLICK || !down.Hit) + return down; + + McpPointerClickResult up = await SendAsync(new McpEcsPointerEventIntent(targetEntityId, aimPoint, button, PointerEventType.PetUp, down.Press)); + + if (!up.UpRayMissed) + return up; // a fresh release, or a failure (scene reload, timeout, preemption) that tells the whole story + + // The release needed the press-frame hit: report the press delivery and flag the divergence. + down.UpRayMissed = true; + + if (!up.Hit) + down.FailureReason = $"the entity disappeared after the press ({up.FailureReason}); only PetDown was delivered"; + + return down; + } + + private UniTask SendAsync(McpEcsPointerEventIntent request) => + McpRequest.SendAsync(world, playerEntity, request, new McpPointerClickResult + { + Hit = false, + FailureReason = "preempted by a newer click_entity call", + }); + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta new file mode 100644 index 00000000000..8b943172934 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ClickEntityTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: d28c7ef4de5c429198677e5f3fb0776d \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs new file mode 100644 index 00000000000..140bbe15a70 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs @@ -0,0 +1,61 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Text; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + public class GetEntityDetailsTool : McpTool + { + private const int MAX_CHARS = 8000; + private const string CURRENT_SCENE = "CURRENT"; + + private readonly IWorldInfoHub worldInfoHub; + + public override string Name => "get_entity_details"; + + public override string Description => + "Dump all components of one entity in the current scene's ECS world (ids come from list_scene_entities)."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Integer("entityId", "Entity id within the current scene world.", isRequired: true); + + public override McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetEntityDetailsTool(IWorldInfoHub worldInfoHub) + { + this.worldInfoHub = worldInfoHub; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetInt("entityId", out int entityId)) + return UniTask.FromResult(McpToolResult.Error("entityId is required.")); + + IWorldInfo? worldInfo = worldInfoHub.WorldInfo(CURRENT_SCENE); + + if (worldInfo == null) + return UniTask.FromResult(McpToolResult.Error("No scene world found at the current parcel.")); + + string dump = worldInfo.EntityComponentsInfo(entityId); + + if (dump.Length <= MAX_CHARS) + return UniTask.FromResult(McpToolResult.Text(dump)); + + var output = new StringBuilder(MAX_CHARS + 64); + output.Append(dump, 0, MAX_CHARS); + output.AppendLine(); + output.Append("... output truncated at ") + .Append(MAX_CHARS) + .Append('/') + .Append(dump.Length) + .Append(" chars"); + + return UniTask.FromResult(McpToolResult.Text(output.ToString())); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta new file mode 100644 index 00000000000..bd9c419498a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetEntityDetailsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 8db7c68630cf74ae7a2fa4153b29875f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs new file mode 100644 index 00000000000..c66f8c2d577 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs @@ -0,0 +1,87 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.Profiles; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + public class GetPlayerStateTool : McpTool + { + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + private readonly ICurrentSceneInfo currentSceneInfo; + + public override string Name => "get_player_state"; + + public override string Description => + "Read the player's current world position, rotation, parcel, velocity and grounded state, the camera position, rotation and mode, " + + "and the wallet address — use the address to tell Explorer instances apart when several run at once."; + + public override JObject OutputSchema => + McpJsonSchema.Object() + .Object("position", JObjectExtensions.VectorSchema()) + .Object("rotationEuler", JObjectExtensions.VectorSchema()) + .Object("parcel", JObjectExtensions.ParcelSchema()) + .Object("velocity", JObjectExtensions.VectorSchema()) + .Boolean("isGrounded") + .Boolean("isPlayerStandingOnScene") + .String("address", "Wallet address of the logged-in player, or null when no profile is loaded.", nullable: true) + .Object("camera", McpJsonSchema.Object() + .Object("position", JObjectExtensions.VectorSchema()) + .Object("rotationEuler", JObjectExtensions.VectorSchema()) + .String("mode") + .Boolean("modeChangeAllowed") + .Boolean("pointerLocked")) + .Build(); + + public override McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetPlayerStateTool(World world, Entity playerEntity, ExposedCameraData exposedCameraData, ICurrentSceneInfo currentSceneInfo) + { + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + this.currentSceneInfo = currentSceneInfo; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + CharacterTransform characterTransform = world.Get(playerEntity); + Vector3 position = characterTransform.Position; + + world.TryGet(playerEntity, out CharacterRigidTransform? rigidTransform); + world.TryGet(playerEntity, out Profile? profile); + + var state = new JObject + { + ["position"] = position.ToVector(), + ["rotationEuler"] = characterTransform.Rotation.eulerAngles.ToVector(), + ["parcel"] = position.ToParcel().ToParcel(), + ["velocity"] = (rigidTransform?.MoveVelocity.Velocity ?? Vector3.zero).ToVector(), + ["isGrounded"] = rigidTransform?.IsGrounded ?? false, + ["isPlayerStandingOnScene"] = currentSceneInfo.IsPlayerStandingOnScene, + ["address"] = profile == null ? JValue.CreateNull() : profile.Compact.UserId, + ["camera"] = new JObject + { + ["position"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["rotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + ["mode"] = McpWireEnum.ToWire(exposedCameraData.CameraMode), + ["modeChangeAllowed"] = SetCameraModeTool.IsModeChangeAllowed(world), + ["pointerLocked"] = exposedCameraData.PointerIsLocked.Value, + }, + }; + + return UniTask.FromResult(McpToolResult.JsonWithStructured(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta new file mode 100644 index 00000000000..c42e70d65de --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetPlayerStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0c34c7751a4c04382810aefd640df2c1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs new file mode 100644 index 00000000000..82459018578 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs @@ -0,0 +1,84 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.Optimization.ThreadSafePool; +using Newtonsoft.Json.Linq; +using System.Collections.Generic; +using System.Text; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetSceneLogsTool : McpTool + { + private enum Severity : byte + { + ALL, + ERROR, + } + + private const int DEFAULT_LIMIT = 100; + private const int MAX_LIMIT = 500; + + // Thread-safe because the tool runs on the transport's thread-pool thread and requests can overlap. + private static readonly ThreadSafeListPool ENTRIES_POOL = new (MAX_LIMIT, 2); + + private readonly SceneLogBuffer logBuffer; + + public override string Name => "get_scene_logs"; + + public override string Description => + "Read the scene's JavaScript console output (logs, warnings, errors and exceptions). Entries carry monotonic sequence numbers; " + + "pass the last seen sequence as sinceSeq to poll incrementally."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Integer("limit", "Maximum entries to return (newest win). Default 100.") + .Enum("severity", "Filter by severity. Default all.") + .Integer("sinceSeq", "Only return entries with a sequence number greater than this."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + // SceneLogBuffer is thread-safe and nothing else here touches ECS/Unity state, so the tool runs on the + // transport's thread-pool thread and answers even while the main thread is busy loading or paused. + public override bool RequiresMainThread => false; + + public GetSceneLogsTool(SceneLogBuffer logBuffer) + { + this.logBuffer = logBuffer; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + + if (!arguments.TryGetEnum("severity", Severity.ALL, out Severity severity)) + return UniTask.FromResult(McpToolResult.Error("severity must be one of: all, error.")); + + bool errorsOnly = severity == Severity.ERROR; + long sinceSeq = arguments.GetLong("sinceSeq", -1); + + using var scope = ENTRIES_POOL.Get(out List entries); + logBuffer.CopyTo(entries, sinceSeq, errorsOnly, limit); + + var output = new StringBuilder(); + + output.Append("latestSeq=") + .Append(logBuffer.LatestSeq) + .Append(" returned=") + .Append(entries.Count) + .AppendLine(); + + foreach (SceneLogBuffer.Entry entry in entries) + { + output.Append('#') + .Append(entry.Seq) + .Append(' ') + .Append(entry.Message) + .AppendLine(); + } + + return UniTask.FromResult(McpToolResult.Text(output.ToString())); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta new file mode 100644 index 00000000000..7e2d718b638 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneLogsTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 0ef4e7ee221c646b1ad9e254b9326614 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs new file mode 100644 index 00000000000..e885fc1addb --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs @@ -0,0 +1,82 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.RealmNavigation; +using ECS.SceneLifeCycle; +using ECS.SceneLifeCycle.CurrentScene; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class GetSceneStateTool : McpTool + { + private readonly IScenesCache scenesCache; + private readonly ICurrentSceneInfo currentSceneInfo; + private readonly ILoadingStatus loadingStatus; + private readonly bool localSceneDevelopment; + + public override string Name => "get_scene_state"; + + public override string Description => + "Read the state of the scene at the player's current parcel: name, base parcel, runtime state (including JavaScript/ECS errors), " + + "readiness, asset loading progress and the global loading-screen stage. Call this after teleporting or reloading before interacting."; + + public override JObject OutputSchema => + McpJsonSchema.Object() + .Object("currentParcel", JObjectExtensions.ParcelSchema()) + .String("loadingStage") + .Boolean("loadingScreenOn") + .Boolean("localSceneDevelopment") + .Object("scene", McpJsonSchema.Object() + .String("name") + .Object("baseParcel", JObjectExtensions.ParcelSchema()) + .String("sdkVersion", "SDK version reported by the scene, or null when unknown.", nullable: true) + .String("state") + .Boolean("isReady") + .Boolean("assetsLoadingConcluded") + .String("runningStatus"), + "The scene at the player's current parcel, or null when no scene is loaded there.", nullable: true) + .Build(); + + public override McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public GetSceneStateTool(IScenesCache scenesCache, ICurrentSceneInfo currentSceneInfo, ILoadingStatus loadingStatus, bool localSceneDevelopment) + { + this.scenesCache = scenesCache; + this.currentSceneInfo = currentSceneInfo; + this.loadingStatus = loadingStatus; + this.localSceneDevelopment = localSceneDevelopment; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + Vector2Int currentParcel = scenesCache.CurrentParcel.Value; + ISceneFacade? scene = scenesCache.CurrentScene.Value; + + var state = new JObject + { + ["currentParcel"] = currentParcel.ToParcel(), + ["loadingStage"] = loadingStatus.CurrentStage.Value.ToString(), + ["loadingScreenOn"] = loadingStatus.IsLoadingScreenOn(), + ["localSceneDevelopment"] = localSceneDevelopment, + ["scene"] = scene == null + ? JValue.CreateNull() + : new JObject + { + ["name"] = scene.Info.Name, + ["baseParcel"] = scene.Info.BaseParcel.ToParcel(), + ["sdkVersion"] = scene.Info.SdkVersion, + ["state"] = scene.SceneStateProvider.State.Value().ToString(), + ["isReady"] = scene.IsSceneReady(), + ["assetsLoadingConcluded"] = scene.SceneData.SceneLoadingConcluded, + ["runningStatus"] = currentSceneInfo.SceneStatus.Value?.ToString() ?? "Unknown", + }, + }; + + return UniTask.FromResult(McpToolResult.JsonWithStructured(state)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta new file mode 100644 index 00000000000..2fdf38209b6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/GetSceneStateTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: f9c6cd0d753384d63b9e283c8cd1eabb \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs new file mode 100644 index 00000000000..e59cca1b95a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs @@ -0,0 +1,99 @@ +using Cysharp.Threading.Tasks; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using SceneRunner.Debugging; +using SceneRunner.Debugging.Hub; +using System.Collections.Generic; +using System.Text; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Lists the entity ids of the current scene's ECS world through the same unsynchronized + /// path the existing debug tooling uses. + /// + public class ListSceneEntitiesTool : McpTool + { + private const int DEFAULT_LIMIT = 200; + private const int MAX_LIMIT = 2000; + private const string CURRENT_SCENE = "CURRENT"; + + private readonly IWorldInfoHub worldInfoHub; + + public override string Name => "list_scene_entities"; + + public override string Description => + "List the ECS entity ids of the scene at the player's current parcel. Feed an id into get_entity_details to inspect its components."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Integer("limit", "Maximum ids to return. Default 200."); + + public override JObject OutputSchema => + McpJsonSchema.Object() + .Integer("total") + .Integer("returned") + .Boolean("truncated") + .IntegerArray("entityIds") + .Build(); + + public override McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public ListSceneEntitiesTool(IWorldInfoHub worldInfoHub) + { + this.worldInfoHub = worldInfoHub; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int limit = Mathf.Clamp(arguments.GetInt("limit", DEFAULT_LIMIT), 1, MAX_LIMIT); + + IWorldInfo? worldInfo = worldInfoHub.WorldInfo(CURRENT_SCENE); + + if (worldInfo == null) + return UniTask.FromResult(McpToolResult.Error("No scene world found at the current parcel.")); + + IReadOnlyList entityIds = worldInfo.EntityIds(); + int returned = Mathf.Min(limit, entityIds.Count); + bool truncated = returned < entityIds.Count; + + var ids = new JArray(); + var output = new StringBuilder(); + output.Append("total=") + .Append(entityIds.Count) + .Append(" returned=") + .Append(returned) + .AppendLine(); + + for (var i = 0; i < entityIds.Count && i < limit; i++) + { + ids.Add(entityIds[i]); + output.Append(entityIds[i]); + output.Append(i < entityIds.Count - 1 && i < limit - 1 ? ", " : string.Empty); + } + + if (truncated) + { + output.AppendLine(); + output.Append(returned) + .Append(" of ") + .Append(entityIds.Count) + .Append(" shown; raise limit (max ") + .Append(MAX_LIMIT) + .Append(") to see the rest."); + } + + var structured = new JObject + { + ["total"] = entityIds.Count, + ["returned"] = returned, + ["truncated"] = truncated, + ["entityIds"] = ids, + }; + + return UniTask.FromResult(McpToolResult.TextWithStructured(output.ToString(), structured)); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta new file mode 100644 index 00000000000..0e7a23b95e6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ListSceneEntitiesTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 76659a159375e46aabcb920885ee30a3 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs new file mode 100644 index 00000000000..2f87bf0502f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs @@ -0,0 +1,61 @@ +using Arch.Core; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + public class LookAtTool : McpTool + { + private readonly IGlobalWorldActions globalWorldActions; + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + + public override string Name => "look_at"; + + public override string Description => + "Rotate the camera to look at a world-space point (x,y,z in meters). Useful to center something on screen before a screenshot."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Number("x", isRequired: true) + .Number("y", isRequired: true) + .Number("z", isRequired: true); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public LookAtTool(IGlobalWorldActions globalWorldActions, World world, Entity playerEntity, ExposedCameraData exposedCameraData) + { + this.globalWorldActions = globalWorldActions; + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates are required."); + + Vector3 playerPosition = world.Get(playerEntity).Position; + globalWorldActions.RotateCamera(new Vector3(x, y, z), playerPosition); + + // Let the Cinemachine systems apply the look-at intent before reading the camera back. + await UniTask.DelayFrame(3, cancellationToken: ct); + + var result = new JObject + { + ["cameraPosition"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["cameraRotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta new file mode 100644 index 00000000000..083259d660a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/LookAtTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: f48ec52210bd7465e84233aa7ae83b13 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs new file mode 100644 index 00000000000..02d65f691b1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs @@ -0,0 +1,82 @@ +using Arch.Core; +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + public class MoveToTool : McpTool + { + private const float MAX_DURATION_SEC = 30f; + private const float COMPLETION_GRACE_SEC = 5f; + + private readonly IGlobalWorldActions globalWorldActions; + private readonly World world; + private readonly Entity playerEntity; + + public override string Name => "move_to"; + + public override string Description => + "Move the player to a world-space position (x,y,z in meters; one parcel is 16x16m). Instant by default, or smooth over durationSec. " + + "Optionally face a look-at target on arrival. For crossing to another scene prefer the teleport tool."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Number("x", isRequired: true) + .Number("y", isRequired: true) + .Number("z", isRequired: true) + .Number("lookAtX") + .Number("lookAtY") + .Number("lookAtZ") + .Number("durationSec", "Seconds to move over; 0 (default) teleports instantly."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public MoveToTool(IGlobalWorldActions globalWorldActions, World world, Entity playerEntity) + { + this.globalWorldActions = globalWorldActions; + this.world = world; + this.playerEntity = playerEntity; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates are required."); + + Vector3? lookAtTarget = null; + + if (arguments.TryGetFloat("lookAtX", out float lookAtX) && arguments.TryGetFloat("lookAtY", out float lookAtY) && arguments.TryGetFloat("lookAtZ", out float lookAtZ)) + lookAtTarget = new Vector3(lookAtX, lookAtY, lookAtZ); + + float durationSec = Mathf.Clamp(arguments.GetFloat("durationSec", 0f), 0f, MAX_DURATION_SEC); + var targetPosition = new Vector3(x, y, z); + + try + { + await globalWorldActions.MoveAndRotatePlayerAsync(targetPosition, lookAtTarget, lookAtTarget, durationSec, ct) + .Timeout(TimeSpan.FromSeconds(durationSec + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) { return McpToolResult.Error($"move_to did not complete within {durationSec + COMPLETION_GRACE_SEC}s."); } + + // Give the teleport/rotation systems a couple of frames to apply the intents before reading back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + Vector3 finalPosition = world.Get(playerEntity).Position; + + var result = new JObject + { + ["position"] = finalPosition.ToVector(), + ["parcel"] = finalPosition.ToParcel().ToParcel(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta new file mode 100644 index 00000000000..789ec935f21 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/MoveToTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: e497faa837a32493bb1f287241cb4661 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs new file mode 100644 index 00000000000..ee18c4d5bd6 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs @@ -0,0 +1,84 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.CharacterMotion.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.SkyBox.Components; +using ECS.SceneLifeCycle; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Reloads the scene at the player's parcel, freezing character motion and the skybox during the reload + /// exactly like the local-scene-development hot reload does. + /// + public class ReloadSceneTool : McpTool + { + private const float MIN_TIMEOUT_SEC = 5f; + private const float MAX_TIMEOUT_SEC = 120f; + private const float DEFAULT_TIMEOUT_SEC = 15f; + + private readonly ECSReloadScene reloadScene; + private readonly IScenesCache scenesCache; + private readonly World world; + private readonly Entity playerEntity; + private readonly Entity skyboxEntity; + + public override string Name => "reload_scene"; + + public override string Description => + "Reload the scene at the player's current parcel and wait for it to restart. Use after editing scene code " + + "when hot reload didn't trigger, or to reset scene state before a test run."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Number("timeoutSec", "Maximum seconds to wait for the reload. Default 15."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: true, idempotent: false); + + public ReloadSceneTool(ECSReloadScene reloadScene, IScenesCache scenesCache, World world, Entity playerEntity, Entity skyboxEntity) + { + this.reloadScene = reloadScene; + this.scenesCache = scenesCache; + this.world = world; + this.playerEntity = playerEntity; + this.skyboxEntity = skyboxEntity; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + if (scenesCache.CurrentScene.Value == null) + return McpToolResult.Error("There is no scene at the current parcel to reload."); + + try + { + world.AddOrGet(playerEntity, new StopCharacterMotion()); + world.AddOrGet(skyboxEntity, new PauseSkyboxTimeUpdate()); + + ISceneFacade? reloadedScene = await reloadScene.TryReloadSceneAsync(ct) + .Timeout(TimeSpan.FromSeconds(timeoutSec)); + + if (reloadedScene == null) + return McpToolResult.Error("Reload failed: no reloadable scene was found at the current parcel."); + } + catch (TimeoutException) + { + return McpToolResult.Error($"Scene reload did not complete within {timeoutSec}s. Check get_scene_state and get_scene_logs."); + } + finally + { + await UniTask.SwitchToMainThread(); + world.Remove(playerEntity); + world.Remove(skyboxEntity); + } + + return McpToolResult.Text("Scene reloaded. Call get_scene_state to confirm readiness and get_scene_logs for startup output."); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta new file mode 100644 index 00000000000..218fe29e9eb --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ReloadSceneTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1ca31906893da454f85da8932ebb45c9 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs new file mode 100644 index 00000000000..9698eef2b45 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs @@ -0,0 +1,270 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Collections; +using System.Threading; +using Unity.Collections; +using UnityEngine; +using UnityEngine.Experimental.Rendering; +using UnityEngine.Rendering; +using Utility; +using Utility.Multithreading; +using Object = UnityEngine.Object; + +namespace DCL.McpServer.Tools +{ + /// + /// Captures the current frame (back buffer including UI, or a world-only camera render), + /// downscales it and returns it as an MCP image content block. Textures never accumulate: + /// temporaries are released per call and the ReadPixels fallback reuses one persistent buffer. + /// + public class ScreenshotTool : McpTool, IDisposable + { + private enum OutputFormat : byte + { + JPG, + PNG, + } + + private const int DEFAULT_MAX_WIDTH = 1280; + private const int MIN_WIDTH = 64; + private const int MAX_WIDTH = 1920; + private const int JPG_QUALITY = 75; + + private const string MIME_TYPE_PNG = "image/png"; + private const string MIME_TYPE_JPEG = "image/jpeg"; + + private readonly ICoroutineRunner coroutineRunner; + private readonly World world; + private readonly Entity playerEntity; + + // Reused across calls by the ReadPixels fallback so repeated captures don't allocate new textures. + private Texture2D? readPixelsBuffer; + + // True while a capture is running; concurrent requests are rejected so one set of buffers suffices. + // A plain bool is safe: tool execution is marshalled onto the main thread by the dispatcher, so the + // check-and-set below runs without preemption before the first await. + private bool capturing; + + public override string Name => "screenshot"; + + public override string Description => + "Capture a screenshot of what the player currently sees in the Explorer, including scene UI. " + + "Use worldOnly to exclude all UI overlays. Returns a downscaled image plus a caption with the capture context."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Integer("maxWidth", "Maximum output width in pixels (aspect ratio preserved). Default 1280.") + .Enum("quality", "Output encoding. Default jpg.") + .Boolean("worldOnly", "Render only the 3D world through the main camera, excluding UI. Default false."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.ReadOnly(); + + public ScreenshotTool(ICoroutineRunner coroutineRunner, World world, Entity playerEntity) + { + this.coroutineRunner = coroutineRunner; + this.world = world; + this.playerEntity = playerEntity; + } + + public void Dispose() + { + if (readPixelsBuffer != null) + Object.Destroy(readPixelsBuffer); + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + int maxWidth = Mathf.Clamp(arguments.GetInt("maxWidth", DEFAULT_MAX_WIDTH), MIN_WIDTH, MAX_WIDTH); + + if (!arguments.TryGetEnum("quality", OutputFormat.JPG, out OutputFormat format)) + return McpToolResult.Error("quality must be one of: jpg, png."); + + bool asPng = format == OutputFormat.PNG; + bool worldOnly = arguments.GetBool("worldOnly", false); + + if (capturing) + return McpToolResult.Error("Another screenshot capture is already in progress; retry when it completes."); + + capturing = true; + + try { return await CaptureAsync(maxWidth, asPng, worldOnly, ct); } + finally { capturing = false; } + } + + private async UniTask CaptureAsync(int maxWidth, bool asPng, bool worldOnly, CancellationToken ct) + { + Texture2D? backbufferCopy = null; + RenderTexture? worldRender = null; + RenderTexture? downscaled = null; + + try + { + int sourceWidth; + int sourceHeight; + + if (worldOnly) + { + Camera camera = world.CacheCamera().GetCameraComponent(world).Camera; + + // URP replaces the camera's internal color buffer with the target texture's descriptor + // (CreateRenderTextureDescriptor), so an LDR target silently downgrades the whole render + // to 8-bit and clamps emissives to 1.0, starving bloom and other HDR-dependent post effects. + // An HDR temporary keeps the pipeline HDR end-to-end; the downscale blit into the sRGB + // descriptor below performs the linear-to-sRGB conversion. + worldRender = RenderTexture.GetTemporary(camera.pixelWidth, camera.pixelHeight, 24, RenderTextureFormat.DefaultHDR); + + // Camera.Render() is unsupported under URP: redirect the camera's output into the + // render texture for exactly one frame instead, then restore it. + RenderTexture? previousTarget = camera.targetTexture; + camera.targetTexture = worldRender; + + try { await WaitForEndOfFrameAsync(ct); } + finally + { + await UniTask.SwitchToMainThread(); + camera.targetTexture = previousTarget; + } + + sourceWidth = worldRender.width; + sourceHeight = worldRender.height; + } + else + { + backbufferCopy = await CaptureBackbufferAsync(ct); + + if (backbufferCopy == null) + return McpToolResult.Error("Back buffer capture failed."); + + sourceWidth = backbufferCopy.width; + sourceHeight = backbufferCopy.height; + } + + int width = Mathf.Min(maxWidth, sourceWidth); + int height = Mathf.Max(1, Mathf.RoundToInt((float)sourceHeight * width / sourceWidth)); + + var descriptor = new RenderTextureDescriptor(width, height) + { + graphicsFormat = OutputGraphicsFormat(), sRGB = true, msaaSamples = 1, depthBufferBits = 0, + mipCount = 1, useMipMap = false, + }; + + downscaled = RenderTexture.GetTemporary(descriptor); + + if (backbufferCopy != null) + { + Graphics.Blit(backbufferCopy, downscaled); + + // The back buffer copy is a fresh Texture2D allocated by Unity on every capture: destroy it as soon as it's blitted. + Object.Destroy(backbufferCopy); + backbufferCopy = null; + } + else + { + Graphics.Blit(worldRender, downscaled); + RenderTexture.ReleaseTemporary(worldRender); + worldRender = null; + } + + byte[] encoded; + string mimeType; + + AsyncGPUReadbackRequest readback = await AsyncGPUReadback.Request(downscaled).WithCancellation(ct); + + if (readback.hasError) { (encoded, mimeType) = EncodeViaReadPixels(downscaled, asPng); } + else + { + NativeArray rawPixels = readback.GetData(); + + using (NativeArray encodedNative = asPng + ? ImageConversion.EncodeNativeArrayToPNG(rawPixels, downscaled.graphicsFormat, (uint)width, (uint)height) + : ImageConversion.EncodeNativeArrayToJPG(rawPixels, downscaled.graphicsFormat, (uint)width, (uint)height)) + encoded = encodedNative.ToArray(); + + mimeType = asPng ? MIME_TYPE_PNG : MIME_TYPE_JPEG; + } + + Vector2Int parcel = world.Get(playerEntity).Position.ToParcel(); + var caption = $"{width}x{height} {(worldOnly ? "world-only" : "full-view")} capture at parcel ({parcel.x},{parcel.y})"; + + // Base64 conversion of the encoded image happens off the main thread. + await DCLTask.SwitchToThreadPool(); + return McpToolResult.Image(encoded, mimeType, caption); + } + finally + { + if (backbufferCopy != null || worldRender != null || downscaled != null) + { + await UniTask.SwitchToMainThread(); + + if (backbufferCopy != null) Object.Destroy(backbufferCopy); + if (worldRender != null) RenderTexture.ReleaseTemporary(worldRender); + if (downscaled != null) RenderTexture.ReleaseTemporary(downscaled); + } + } + } + + private UniTask CaptureBackbufferAsync(CancellationToken ct) + { + var completion = new UniTaskCompletionSource(); + coroutineRunner.StartCoroutine(CaptureBackbufferCoroutine(completion)); + return completion.Task.AttachExternalCancellation(ct); + } + + private UniTask WaitForEndOfFrameAsync(CancellationToken ct) + { + var completion = new UniTaskCompletionSource(); + coroutineRunner.StartCoroutine(SignalEndOfFrameCoroutine(completion)); + return completion.Task.AttachExternalCancellation(ct); + } + + private static IEnumerator SignalEndOfFrameCoroutine(UniTaskCompletionSource completion) + { + yield return GameObjectExtensions.WAIT_FOR_END_OF_FRAME; + completion.TrySetResult(); + } + + private static IEnumerator CaptureBackbufferCoroutine(UniTaskCompletionSource completion) + { + // The back buffer is only complete (UI included) at the very end of the frame. + yield return GameObjectExtensions.WAIT_FOR_END_OF_FRAME; + + Texture2D texture = ScreenCapture.CaptureScreenshotAsTexture(); + + if (!completion.TrySetResult(texture)) + Object.Destroy(texture); + } + + private (byte[] bytes, string mimeType) EncodeViaReadPixels(RenderTexture source, bool asPng) + { + if (readPixelsBuffer == null) + readPixelsBuffer = new Texture2D(source.width, source.height, TextureFormat.RGBA32, false); + else if (readPixelsBuffer.width != source.width || readPixelsBuffer.height != source.height) + readPixelsBuffer.Reinitialize(source.width, source.height, readPixelsBuffer.graphicsFormat, false); + + RenderTexture? previousActive = RenderTexture.active; + RenderTexture.active = source; + readPixelsBuffer.ReadPixels(new Rect(0, 0, source.width, source.height), 0, 0); + readPixelsBuffer.Apply(); + RenderTexture.active = previousActive; + + return asPng + ? (readPixelsBuffer.EncodeToPNG(), MIME_TYPE_PNG) + : (readPixelsBuffer.EncodeToJPG(JPG_QUALITY), MIME_TYPE_JPEG); + } + + private static GraphicsFormat OutputGraphicsFormat() + { + var preferred = GraphicsFormat.R8G8B8A8_SRGB; + + if (SystemInfo.IsFormatSupported(preferred, GraphicsFormatUsage.Render)) + return preferred; + + return SystemInfo.GetCompatibleFormat(preferred, GraphicsFormatUsage.Render); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta new file mode 100644 index 00000000000..7c72a9a5cc9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/ScreenshotTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 781cfdc63c1c5405aa87fa01a82e4a4f \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs new file mode 100644 index 00000000000..72cf71c8ddb --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs @@ -0,0 +1,52 @@ +using Cysharp.Threading.Tasks; +using DCL.Chat.History; +using DCL.Chat.MessageBus; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Sends a message to the Nearby chat channel. Messages starting with '/' run through the chat + /// command pipeline, so agents can drive commands the same way a user typing in chat would. + /// + public class SendChatTool : McpTool + { + private const int MAX_MESSAGE_LENGTH = 500; + + private readonly IChatMessagesBus chatMessagesBus; + + public override string Name => "send_chat"; + + public override string Description => + "Send a message to the Nearby chat channel. Messages starting with '/' run chat commands " + + "(e.g. /goto x,y, /reload, /help); command output appears in chat and scene logs."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.String("message", "The chat message or /command to send.", isRequired: true); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public SendChatTool(IChatMessagesBus chatMessagesBus) + { + this.chatMessagesBus = chatMessagesBus; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + string message = arguments.GetString("message", string.Empty); + + if (string.IsNullOrWhiteSpace(message)) + return UniTask.FromResult(McpToolResult.Error("message is required.")); + + if (message.Length > MAX_MESSAGE_LENGTH) + return UniTask.FromResult(McpToolResult.Error($"message exceeds the {MAX_MESSAGE_LENGTH} character limit.")); + + chatMessagesBus.SendWithUtcNowTimestamp(ChatChannel.NEARBY_CHANNEL, message, ChatMessageOrigin.CHAT); + + return UniTask.FromResult(McpToolResult.Text($"Sent to Nearby: {message}")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta new file mode 100644 index 00000000000..7da0103ce84 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SendChatTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 9e754e2afb7734afb8f6b5e0f735b168 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs new file mode 100644 index 00000000000..d3db59f81f0 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs @@ -0,0 +1,99 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.CharacterCamera; +using DCL.InWorldCamera; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using ECS.Abstract; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Switches the camera mode by writing (the same pattern scene systems use; + /// ControlCinemachineVirtualCameraSystem applies it next frame). The direct write bypasses the user-input + /// gates, so this tool re-checks them itself and refuses when a scene holds the camera. + /// + public class SetCameraModeTool : McpTool + { + private static readonly CameraMode[] ALLOWED_MODES = { CameraMode.FirstPerson, CameraMode.ThirdPerson, CameraMode.DroneView, CameraMode.Free }; + + private readonly World world; + private readonly ExposedCameraData exposedCameraData; + + public override string Name => "set_camera_mode"; + + public override string Description => + "Switch the player camera mode (first_person, third_person, drone_view, or the free-fly camera), like a user pressing the camera key. " + + "Refuses with an explanation when the scene locks the mode (CameraModeArea, scene virtual camera, photo camera). " + + "Any player movement drops free back to third_person."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Enum("mode", "Target camera mode.", ALLOWED_MODES, isRequired: true); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public SetCameraModeTool(World world, ExposedCameraData exposedCameraData) + { + this.world = world; + this.exposedCameraData = exposedCameraData; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetEnum("mode", out CameraMode targetMode, ALLOWED_MODES)) + return McpToolResult.Error("mode must be one of: first_person, third_person, drone_view, free."); + + string? blockReason = TrySwitchMode(world, targetMode, out CameraMode previousMode); + + if (blockReason != null) + return McpToolResult.Error(blockReason); + + // Let ControlCinemachineVirtualCameraSystem activate the matching virtual camera before reading back. + await UniTask.DelayFrame(2, cancellationToken: ct); + + var result = new JObject + { + ["requestedMode"] = McpWireEnum.ToWire(targetMode), + ["currentMode"] = McpWireEnum.ToWire(exposedCameraData.CameraMode), + ["previousMode"] = McpWireEnum.ToWire(previousMode), + }; + + return McpToolResult.Json(result); + } + + /// + /// Same gates ControlCinemachineVirtualCameraSystem.HandleCameraInput applies to user input. + /// Main thread only. + /// + internal static bool IsModeChangeAllowed(World world) + { + SingleInstanceEntity cameraEntity = world.CacheCamera(); + ref readonly CameraComponent camera = ref cameraEntity.GetCameraComponent(world); + + return camera.Mode != CameraMode.SDKCamera + && camera.CameraInputChangeEnabled + && !world.Has(cameraEntity); + } + + internal static string? TrySwitchMode(World world, CameraMode targetMode, out CameraMode previousMode) + { + SingleInstanceEntity cameraEntity = world.CacheCamera(); + ref CameraComponent camera = ref cameraEntity.GetCameraComponent(world); + previousMode = camera.Mode; + + if (camera.Mode == CameraMode.SDKCamera) + return "A scene virtual camera controls the view right now (mode SDKCamera); the mode cannot change until the scene releases it."; + + if (!camera.CameraInputChangeEnabled) + return $"Camera mode is locked by the scene (CameraModeArea; current mode: {camera.Mode}). Leave the area to change modes."; + + if (world.Has(cameraEntity)) + return "The in-world photo camera is active; close it before changing the camera mode."; + + camera.Mode = targetMode; + return null; + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta new file mode 100644 index 00000000000..f5b3533b96f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraModeTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 297288bb84c8d4cd1af0af018a8a4123 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs new file mode 100644 index 00000000000..efe6d6a6187 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs @@ -0,0 +1,138 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterCamera; +using DCL.CharacterCamera.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using ECS.Abstract; +using Newtonsoft.Json.Linq; +using System.Threading; +using UnityEngine; +using Utility.Arch; + +namespace DCL.McpServer.Tools +{ + /// + /// Places the free camera at an absolute world position, optionally aiming it and setting its FOV. + /// Enters Free mode when needed (same scene-lock gates as ) and waits + /// for the Cinemachine blend to reach the target before reporting the actual pose. + /// + public class SetCameraPoseTool : McpTool + { + private const float DEFAULT_TIMEOUT_SEC = 5f; + private const float MIN_TIMEOUT_SEC = 0.5f; + private const float MAX_TIMEOUT_SEC = 15f; + private const float SETTLE_EPSILON = 0.1f; + private const int POLL_INTERVAL_MS = 100; + private const float MIN_FOV = 10f; + private const float MAX_FOV = 120f; + private const int CAMERA_APPLY_DELAY_FRAMES = 2; + + private readonly World world; + private readonly Entity playerEntity; + private readonly ExposedCameraData exposedCameraData; + + public override string Name => "set_camera_pose"; + + public override string Description => + "Place the free camera at an absolute world position, optionally aiming it at a point and setting its field of view. " + + "Enters the free camera mode if needed (refuses with the reason when the scene locks the camera). The camera stays " + + "put while the player moves; restore a player-following view with set_camera_mode third_person."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Number("x", "Camera world position.", isRequired: true) + .Number("y", isRequired: true) + .Number("z", isRequired: true) + .Number("lookAtX", "Optional world point to aim at (all three lookAt components required together).") + .Number("lookAtY") + .Number("lookAtZ") + .Number("fov", "Optional vertical field of view in degrees (10-120).") + .Number("timeoutSec", "Seconds to wait for the camera to settle at the target. Default 5, max 15."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public SetCameraPoseTool(World world, Entity playerEntity, ExposedCameraData exposedCameraData) + { + this.world = world; + this.playerEntity = playerEntity; + this.exposedCameraData = exposedCameraData; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetFloat("x", out float x) || !arguments.TryGetFloat("y", out float y) || !arguments.TryGetFloat("z", out float z)) + return McpToolResult.Error("x, y and z world coordinates for the camera position are required."); + + bool hasLookAtX = arguments.TryGetFloat("lookAtX", out float lookAtX); + bool hasLookAtY = arguments.TryGetFloat("lookAtY", out float lookAtY); + bool hasLookAtZ = arguments.TryGetFloat("lookAtZ", out float lookAtZ); + bool hasLookAt = hasLookAtX && hasLookAtY && hasLookAtZ; + + if ((hasLookAtX || hasLookAtY || hasLookAtZ) && !hasLookAt) + return McpToolResult.Error("lookAtX, lookAtY and lookAtZ must be provided together."); + + float? fov = null; + + if (arguments.TryGetFloat("fov", out float fovValue)) + fov = Mathf.Clamp(fovValue, MIN_FOV, MAX_FOV); + + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + var targetPosition = new Vector3(x, y, z); + + SingleInstanceEntity cameraEntity = world.CacheCamera(); + + if (cameraEntity.GetCameraComponent(world).Mode != CameraMode.Free) + { + string? blockReason = SetCameraModeTool.TrySwitchMode(world, CameraMode.Free, out CameraMode _); + + if (blockReason != null) + return McpToolResult.Error(blockReason); + + // Let ControlCinemachineVirtualCameraSystem activate the free vcam (and apply its default + // spawn position, which the pose below overrides). + await UniTask.DelayFrame(CAMERA_APPLY_DELAY_FRAMES, cancellationToken: ct); + } + + if (!world.TryGet(cameraEntity, out ICinemachinePreset? cinemachinePreset) || cinemachinePreset == null) + return McpToolResult.Error("The camera rig is not initialized yet."); + + cinemachinePreset.ForceFreeCameraPose(targetPosition, fov); + + if (hasLookAt) + { + Vector3 playerPosition = world.Get(playerEntity).Position; + world.AddOrSet(cameraEntity, new CameraLookAtIntent(new Vector3(lookAtX, lookAtY, lookAtZ), playerPosition)); + } + + // Entering Free blends the output camera toward the vcam over a couple of seconds; when the + // mode was already Free the pose applies instantly and the first poll succeeds. + var settled = false; + float deadline = UnityEngine.Time.realtimeSinceStartup + timeoutSec; + + while (UnityEngine.Time.realtimeSinceStartup < deadline) + { + if (Vector3.Distance(exposedCameraData.WorldPosition.Value, targetPosition) <= SETTLE_EPSILON) + { + settled = true; + break; + } + + await UniTask.Delay(POLL_INTERVAL_MS, cancellationToken: ct); + } + + // Let the look-at intent apply before reading the rotation back. + await UniTask.DelayFrame(CAMERA_APPLY_DELAY_FRAMES, cancellationToken: ct); + + var result = new JObject + { + ["position"] = exposedCameraData.WorldPosition.Value.ToVector(), + ["rotationEuler"] = exposedCameraData.WorldRotation.Value.eulerAngles.ToVector(), + ["mode"] = McpWireEnum.ToWire(exposedCameraData.CameraMode), + ["settled"] = settled, + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta new file mode 100644 index 00000000000..5ad7c5b677f --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/SetCameraPoseTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 5d66df6294124c629fa19535681000e1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs new file mode 100644 index 00000000000..b3a4e8f6908 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs @@ -0,0 +1,91 @@ +using Cysharp.Threading.Tasks; +using DCL.Chat.Commands; +using DCL.Chat.History; +using DCL.Chat.MessageBus; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using DCL.RealmNavigation; +using ECS.SceneLifeCycle; +using Newtonsoft.Json.Linq; +using SceneRunner.Scene; +using System.Threading; +using UnityEngine; + +namespace DCL.McpServer.Tools +{ + /// + /// Teleports through the same /goto command pipeline a user teleport takes (loading screen included), + /// then polls until the destination scene is ready or the timeout elapses. + /// + public class TeleportTool : McpTool + { + private const int POLL_INTERVAL_MS = 500; + private const float MIN_TIMEOUT_SEC = 5f; + private const float MAX_TIMEOUT_SEC = 300f; + private const float DEFAULT_TIMEOUT_SEC = 60f; + + private readonly IChatMessagesBus chatMessagesBus; + private readonly IScenesCache scenesCache; + private readonly ILoadingStatus loadingStatus; + + public override string Name => "teleport"; + + public override string Description => + "Teleport the player to a parcel (x,y) through the regular /goto flow and wait until the destination scene is ready. " + + "Reports the final scene state; follow up with get_scene_state for details."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Integer("x", "Target parcel X coordinate.", isRequired: true) + .Integer("y", "Target parcel Y coordinate.", isRequired: true) + .Boolean("waitForReady", "Wait until the destination scene is ready. Default true.") + .Number("timeoutSec", "Maximum seconds to wait for readiness. Default 60."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: true); + + public TeleportTool(IChatMessagesBus chatMessagesBus, IScenesCache scenesCache, ILoadingStatus loadingStatus) + { + this.chatMessagesBus = chatMessagesBus; + this.scenesCache = scenesCache; + this.loadingStatus = loadingStatus; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + if (!arguments.TryGetInt("x", out int x) || !arguments.TryGetInt("y", out int y)) + return McpToolResult.Error("Both x and y parcel coordinates are required."); + + bool waitForReady = arguments.GetBool("waitForReady", true); + float timeoutSec = Mathf.Clamp(arguments.GetFloat("timeoutSec", DEFAULT_TIMEOUT_SEC), MIN_TIMEOUT_SEC, MAX_TIMEOUT_SEC); + + chatMessagesBus.SendWithUtcNowTimestamp(ChatChannel.NEARBY_CHANNEL, $"/{ChatCommandsUtils.COMMAND_GOTO} {x},{y}", ChatMessageOrigin.RESTRICTED_ACTION_API); + + if (!waitForReady) + return McpToolResult.Text($"Teleport to ({x},{y}) requested."); + + var targetParcel = new Vector2Int(x, y); + float deadline = UnityEngine.Time.realtimeSinceStartup + timeoutSec; + + while (UnityEngine.Time.realtimeSinceStartup < deadline) + { + await UniTask.Delay(POLL_INTERVAL_MS, cancellationToken: ct); + + ISceneFacade? currentScene = scenesCache.CurrentScene.Value; + bool arrived = scenesCache.CurrentParcel.Value == targetParcel || (currentScene?.Contains(targetParcel) ?? false); + + if (!arrived || loadingStatus.IsLoadingScreenOn()) + continue; + + if (currentScene == null) + return McpToolResult.Text($"Arrived at ({x},{y}); no scene is deployed at this parcel."); + + if (currentScene.SceneStateProvider.IsNotRunningState()) + return McpToolResult.Error($"Arrived at ({x},{y}) but scene '{currentScene.Info.Name}' is not running: {currentScene.SceneStateProvider.State.Value()}. Check get_scene_logs."); + + if (currentScene.IsSceneReady()) + return McpToolResult.Text($"Teleported to ({x},{y}). Scene '{currentScene.Info.Name}' is ready."); + } + + return McpToolResult.Error($"Teleport to ({x},{y}) did not reach a ready scene within {timeoutSec}s. Check get_scene_state."); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta new file mode 100644 index 00000000000..094575a2b3d --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TeleportTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 1f263b4ea6e6e489a83b9b5accd70c06 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs new file mode 100644 index 00000000000..6fe534bcbd5 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs @@ -0,0 +1,56 @@ +using CrdtEcsBridge.RestrictedActions; +using Cysharp.Threading.Tasks; +using DCL.ECSComponents; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System.Threading; + +namespace DCL.McpServer.Tools +{ + /// + /// Plays or stops an avatar emote through , the same intent path + /// a scene uses, so agents can verify emote-driven behaviour without touching the avatar directly. + /// + public class TriggerEmoteTool : McpTool + { + private readonly IGlobalWorldActions globalWorldActions; + + public override string Name => "trigger_emote"; + + public override string Description => + "Play an avatar emote by URN (e.g. a base emote like 'wave', 'dance', 'clap'), or stop the current one with stop: true."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.String("urn", "Emote URN or base emote id (wave, dance, clap...).") + .Boolean("loop", "Loop the emote until stopped. Default false.") + .Boolean("stop", "Stop the currently playing emote instead of triggering one."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public TriggerEmoteTool(IGlobalWorldActions globalWorldActions) + { + this.globalWorldActions = globalWorldActions; + } + + public override UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + bool stop = arguments.GetBool("stop", false); + string urn = arguments.GetString("urn", string.Empty); + + if (!stop && string.IsNullOrEmpty(urn)) + return UniTask.FromResult(McpToolResult.Error("urn is required (or pass stop: true).")); + + if (stop) + { + globalWorldActions.StopEmote(); + return UniTask.FromResult(McpToolResult.Text("Emote stopped.")); + } + + bool loop = arguments.GetBool("loop", false); + globalWorldActions.TriggerEmote(urn, loop, AvatarEmoteMask.AemFullBody); + + return UniTask.FromResult(McpToolResult.Text($"Emote '{urn}' triggered{(loop ? " (looping)" : string.Empty)}.")); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta new file mode 100644 index 00000000000..f77e3e314b1 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/TriggerEmoteTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 538f89c5fefa643f3936739de46a83e1 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs new file mode 100644 index 00000000000..a249c82e9be --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs @@ -0,0 +1,104 @@ +using Arch.Core; +using Cysharp.Threading.Tasks; +using DCL.Character.Components; +using DCL.CharacterMotion.Components; +using DCL.McpServer.Components; +using DCL.McpServer.Core; +using DCL.McpServer.Utils; +using Newtonsoft.Json.Linq; +using System; +using System.Threading; +using UnityEngine; +using Utility; + +namespace DCL.McpServer.Tools +{ + /// + /// Holds a movement input on the player for a duration via , + /// exercising the regular locomotion pipeline (velocity, collisions, jumps) instead of teleporting. + /// + public class WalkTool : McpTool + { + private const float MIN_SECONDS = 0.1f; + private const float MAX_SECONDS = 30f; + private const float COMPLETION_GRACE_SEC = 5f; + + private static readonly MovementKind[] ALLOWED_KINDS = { MovementKind.WALK, MovementKind.JOG, MovementKind.RUN }; + + private readonly World world; + private readonly Entity playerEntity; + + public override string Name => "walk"; + + public override string Description => + "Walk/jog/run the player in a camera-relative direction for a number of seconds through the real locomotion pipeline " + + "(collisions apply). directionY is forward, directionX is strafe right. Returns the start and end positions."; + + protected override McpJsonSchema DescribeInput(McpJsonSchema schema) => + schema.Number("directionX", "Strafe axis: 1 right, -1 left.", isRequired: true) + .Number("directionY", "Forward axis: 1 forward, -1 backward.", isRequired: true) + .Number("seconds", "How long to hold the movement. Default 1, max 30.") + .Enum("kind", "Movement speed. Default jog.", ALLOWED_KINDS) + .Boolean("jump", "Jump once at the start of the movement. Default false."); + + public override McpToolAnnotations Annotations => McpToolAnnotations.Mutating(destructive: false, idempotent: false); + + public WalkTool(World world, Entity playerEntity) + { + this.world = world; + this.playerEntity = playerEntity; + } + + public override async UniTask ExecuteAsync(JObject arguments, CancellationToken ct) + { + var direction = new Vector2(arguments.GetFloat("directionX", 0f), arguments.GetFloat("directionY", 0f)); + + if (direction == Vector2.zero) + return McpToolResult.Error("directionX and directionY must not both be zero."); + + direction.Normalize(); + + float seconds = Mathf.Clamp(arguments.GetFloat("seconds", 1f), MIN_SECONDS, MAX_SECONDS); + bool jump = arguments.GetBool("jump", false); + + if (!arguments.TryGetEnum("kind", MovementKind.JOG, out MovementKind kind, ALLOWED_KINDS)) + return McpToolResult.Error("kind must be one of: walk, jog, run."); + + Vector3 startPosition = world.Get(playerEntity).Position; + + // A newer walk preempts a pending one; the preempted awaiter completes as a finished (shortened) hold. + UniTask hold = McpRequest.SendAsync(world, playerEntity, new McpEcsMovementOverride + { + Axes = direction, + Kind = kind, + EndTime = UnityEngine.Time.time + seconds, + JumpRequested = jump, + }, AsyncUnit.Default); + + try + { + await hold.AttachExternalCancellation(ct) + .Timeout(TimeSpan.FromSeconds(seconds + COMPLETION_GRACE_SEC)); + } + catch (TimeoutException) + { + await McpRequest.AbandonAsync(world, playerEntity); + return McpToolResult.Error($"walk did not complete within {seconds + COMPLETION_GRACE_SEC}s (is the simulation paused?)."); + } + + await UniTask.SwitchToMainThread(ct); + + Vector3 endPosition = world.Get(playerEntity).Position; + + var result = new JObject + { + ["startPosition"] = startPosition.ToVector(), + ["endPosition"] = endPosition.ToVector(), + ["distance"] = Math.Round(Vector3.Distance(startPosition, endPosition), 2), + ["parcel"] = endPosition.ToParcel().ToParcel(), + }; + + return McpToolResult.Json(result); + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta new file mode 100644 index 00000000000..797981b32b9 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Tools/WalkTool.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: c981a464b2fe440749be02bc2b01d6ff \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils.meta b/Explorer/Assets/DCL/McpServer/Utils.meta new file mode 100644 index 00000000000..41ef52ed9b8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils.meta @@ -0,0 +1,3 @@ +fileFormatVersion: 2 +guid: c2c6933e4faa4a0c9bdb0f363315b0eb +timeCreated: 1784277704 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs new file mode 100644 index 00000000000..e01577bd62a --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs @@ -0,0 +1,110 @@ +using DCL.McpServer.Core; +using Newtonsoft.Json.Linq; +using System; +using UnityEngine; + +namespace DCL.McpServer.Utils +{ + /// + /// Builders for the JSON fragments shared by tool outputs. + /// + public static class JObjectExtensions + { + public static JObject ToVector(this Vector3 value) => + new () + { + ["x"] = Math.Round(value.x, 2), + ["y"] = Math.Round(value.y, 2), + ["z"] = Math.Round(value.z, 2), + }; + + public static JObject ToParcel(this Vector2Int value) => + new () + { + ["x"] = value.x, + ["y"] = value.y, + }; + + /// Output-schema counterpart of — an { x, y, z } object of numbers. + public static McpJsonSchema VectorSchema() => + McpJsonSchema.Object() + .Number("x") + .Number("y") + .Number("z"); + + /// Output-schema counterpart of — an { x, y } object of integers. + public static McpJsonSchema ParcelSchema() => + McpJsonSchema.Object() + .Integer("x") + .Integer("y"); + + public static bool GetBool(this JObject arguments, string name, bool defaultValue) => + arguments[name]?.Type == JTokenType.Boolean ? arguments[name]!.Value() : defaultValue; + + public static int GetInt(this JObject arguments, string name, int defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static long GetLong(this JObject arguments, string name, long defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static float GetFloat(this JObject arguments, string name, float defaultValue) => + arguments[name].IsNumber() ? arguments[name]!.Value() : defaultValue; + + public static string GetString(this JObject arguments, string name, string defaultValue) => + arguments[name]?.Type == JTokenType.String ? arguments[name]!.Value()! : defaultValue; + + /// + /// Reads an enum argument sent as its wire name (see ). False when the + /// argument is missing, not a string, not a member, or outside . + /// + public static bool TryGetEnum(this JObject arguments, string name, out T value, T[]? allowed = null) where T : struct, Enum + { + if (arguments[name]?.Type == JTokenType.String + && McpWireEnum.TryParse(arguments[name]!.Value()!, out value) + && (allowed == null || Array.IndexOf(allowed, value) >= 0)) + return true; + + value = default(T); + return false; + } + + /// Same as , but a missing argument yields instead of failing. + public static bool TryGetEnum(this JObject arguments, string name, T defaultValue, out T value, T[]? allowed = null) where T : struct, Enum + { + if (arguments[name] == null) + { + value = defaultValue; + return true; + } + + return arguments.TryGetEnum(name, out value, allowed); + } + + public static bool TryGetFloat(this JObject arguments, string name, out float value) + { + if (arguments[name].IsNumber()) + { + value = arguments[name]!.Value(); + return true; + } + + value = 0f; + return false; + } + + public static bool TryGetInt(this JObject arguments, string name, out int value) + { + if (arguments[name].IsNumber()) + { + value = arguments[name]!.Value(); + return true; + } + + value = 0; + return false; + } + + private static bool IsNumber(this JToken? token) => + token?.Type is JTokenType.Integer or JTokenType.Float; + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta new file mode 100644 index 00000000000..1a87815e6b8 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/JObjectExtensions.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 139ddf8ed9933484c92df485a5843199 \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs new file mode 100644 index 00000000000..54d315b8a96 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs @@ -0,0 +1,82 @@ +using DCL.UI.DebugMenu.LogHistory; +using System.Collections.Generic; + +namespace DCL.McpServer.Utils +{ + /// + /// Thread-safe ring buffer of scene console log entries with monotonic sequence numbers, + /// so agents can poll incrementally via sinceSeq. + /// + public class SceneLogBuffer + { + private const int CAPACITY = 1000; + + private readonly object gate = new (); + private readonly Entry[] entries = new Entry[CAPACITY]; + + private long nextSeq; + + /// + /// Sequence number of the newest stored entry, or -1 when empty. + /// + public long LatestSeq + { + get + { + lock (gate) { return nextSeq - 1; } + } + } + + /// + /// Fed from ; may be invoked from any thread. + /// + public void Append(DebugMenuConsoleLogEntry logEntry) + { + lock (gate) + { + entries[nextSeq % CAPACITY] = new Entry(nextSeq, logEntry.Type, logEntry.Message); + nextSeq++; + } + } + + /// + /// Copies up to newest entries with Seq greater than + /// into in chronological order. + /// + public void CopyTo(List target, long sinceSeq, bool errorsOnly, int limit) + { + lock (gate) + { + long oldestAvailable = nextSeq >= CAPACITY ? nextSeq - CAPACITY : 0; + long from = sinceSeq + 1 > oldestAvailable ? sinceSeq + 1 : oldestAvailable; + + for (long seq = from; seq < nextSeq; seq++) + { + Entry entry = entries[seq % CAPACITY]; + + if (errorsOnly && entry.Type != LogMessageType.Error) + continue; + + target.Add(entry); + } + + if (target.Count > limit) + target.RemoveRange(0, target.Count - limit); + } + } + + public readonly struct Entry + { + public readonly long Seq; + public readonly LogMessageType Type; + public readonly string Message; + + public Entry(long seq, LogMessageType type, string message) + { + Seq = seq; + Type = type; + Message = message; + } + } + } +} diff --git a/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta new file mode 100644 index 00000000000..4bf93032e48 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/Utils/SceneLogBuffer.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 91d8bd24c95a3413c800ab47ed79d7eb \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/csc.rsp b/Explorer/Assets/DCL/McpServer/csc.rsp new file mode 100644 index 00000000000..dcc377f8979 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/csc.rsp @@ -0,0 +1 @@ +-nullable:enable \ No newline at end of file diff --git a/Explorer/Assets/DCL/McpServer/csc.rsp.meta b/Explorer/Assets/DCL/McpServer/csc.rsp.meta new file mode 100644 index 00000000000..fe6eff8c6c5 --- /dev/null +++ b/Explorer/Assets/DCL/McpServer/csc.rsp.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: cd2ec6b45118083cc5af65714bb6a560 +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs index 017891b0591..7e7fcbbd5bf 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/Container/DiagnosticsContainer.cs @@ -13,11 +13,21 @@ public class DiagnosticsContainer : IDisposable { private const int DEFAULT_REPORT_HANDLERS_COUNT = 2; // DebugLog + Sentry - private ILogHandler defaultLogHandler; - public ReportHubLogger ReportHubLogger { get; private set; } - public SentryReportHandler? Sentry { get; private set; } - public IReportsHandlingSettings Settings { get; private set; } - public SentrySampler? SentrySampler { get; private set; } + private readonly ILogHandler defaultLogHandler; + private readonly ReportHubLogger reportHubLogger; + + public SentryReportHandler? Sentry { get; } + public IReportsHandlingSettings Settings { get; } + public SentrySampler? SentrySampler { get; } + + private DiagnosticsContainer(ILogHandler defaultLogHandler, ReportHubLogger reportHubLogger, SentryReportHandler? sentry, IReportsHandlingSettings settings, SentrySampler? sentrySampler) + { + this.defaultLogHandler = defaultLogHandler; + this.reportHubLogger = reportHubLogger; + Sentry = sentry; + Settings = settings; + SentrySampler = sentrySampler; + } public void Dispose() { @@ -57,14 +67,14 @@ public static DiagnosticsContainer Create(IReportsHandlingSettings settings, par // Enable Hub static accessors ReportHub.Initialize(logger); - return new DiagnosticsContainer { ReportHubLogger = logger, defaultLogHandler = defaultLogHandler, Sentry = sentryReportHandler, Settings = settings, SentrySampler = sentrySampler }; + return new DiagnosticsContainer(defaultLogHandler, logger, sentryReportHandler, settings, sentrySampler); } public void AddDebugConsoleHandler(DebugMenuConsoleLogEntryBus sceneDebugConsoleMessageBus) { SceneDebugConsoleReportHandler reportHandler = AddDebugConsoleReportHandler(sceneDebugConsoleMessageBus); ReportHub.EnforceUnconditionalVerboseLogs = true; - ReportHubLogger.AddHandler(reportHandler); + reportHubLogger.AddHandler(reportHandler); } private static SceneDebugConsoleReportHandler AddDebugConsoleReportHandler(DebugMenuConsoleLogEntryBus sceneDebugConsoleMessageBus) @@ -106,9 +116,11 @@ private static SceneDebugConsoleReportHandler AddDebugConsoleReportHandler(Debug ReportCategory.SCENE_FETCH_REQUEST, ReportCategory.PORTABLE_EXPERIENCE, ReportCategory.EMOTE, + ReportCategory.MCP, }, logType: false); entries.Add(new CategorySeverityMatrix.Entry { Category = ReportCategory.JAVASCRIPT, Severity = LogType.Log }); + entries.Add(new CategorySeverityMatrix.Entry { Category = ReportCategory.MCP, Severity = LogType.Log }); jsOnlyMatrix.entries = entries; return new SceneDebugConsoleReportHandler(jsOnlyMatrix, sceneDebugConsoleMessageBus, false); diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs index b79da65000f..7f81ec9a86e 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportCategory.cs @@ -1,6 +1,4 @@ -using UnityEngine; - -namespace DCL.Diagnostics +namespace DCL.Diagnostics { /// /// Contains names of report categories, IDs must be constant so they can be specified in the attribute @@ -360,6 +358,11 @@ public static class ReportCategory public const string CHROME_DEVTOOL_PROTOCOL = nameof(CHROME_DEVTOOL_PROTOCOL); + /// + /// Embedded MCP (Model Context Protocol) automation server. + /// + public const string MCP = nameof(MCP); + public const string SCENE_PERMISSIONS = nameof(SCENE_PERMISSIONS); public const string MVC_STATE_MACHINE = nameof(MVC_STATE_MACHINE); diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset index c69c1771a1c..3feccb406d8 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsDevelopment.asset @@ -708,6 +708,16 @@ MonoBehaviour: Severity: 2 - Category: MULTIPLAYER Severity: 2 + - Category: MCP + Severity: 3 + - Category: MCP + Severity: 0 + - Category: MCP + Severity: 4 + - Category: MCP + Severity: 1 + - Category: MCP + Severity: 2 sentryMatrix: entries: [] debounceEnabled: 1 diff --git a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset index 57f1e67ae01..93e8412a007 100644 --- a/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset +++ b/Explorer/Assets/DCL/PerformanceAndDiagnostics/Diagnostics/ReportsHandling/ReportsHandlingSettingsProduction.asset @@ -448,6 +448,16 @@ MonoBehaviour: Severity: 0 - Category: MULTIPLAYER Severity: 2 + - Category: MCP + Severity: 4 + - Category: MCP + Severity: 1 + - Category: MCP + Severity: 3 + - Category: MCP + Severity: 2 + - Category: MCP + Severity: 0 - Category: AUTHENTICATION Severity: 3 - Category: AUTHENTICATION diff --git a/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef b/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef index c8338aaecc6..2032d9a6b31 100644 --- a/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef +++ b/Explorer/Assets/DCL/PluginSystem/DCL.Plugins.asmdef @@ -70,10 +70,10 @@ "GUID:451516970bc990e418454aa78b72586e", "GUID:13c468c9ecfc44f249fb8fb69d3365ef", "GUID:a42927d1d4a3b4cda9b076a7adecb9cc", - "GUID:f8127c6ac263abf468221dcbccbde182", "GUID:63cde67b4a5d47449392dfc49fc0c3ed", "GUID:267f743eefeddd44982a713909628102", "GUID:0e1edbec885c4350bb2ef81c6a413df5", + "GUID:f254b2009e854709a0ef0022a7ca697c", "GUID:3c1a9be47d0f42e2a5b8c6d4e9f01a27" ], "includePlatforms": [], @@ -98,4 +98,4 @@ } ], "noEngineReferences": false -} +} \ No newline at end of file diff --git a/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef b/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef index f22886fcbd1..9b998b9b1c7 100644 --- a/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef +++ b/Explorer/Assets/DCL/Tests/Editor/DCL.EditMode.Tests.asmdef @@ -61,7 +61,8 @@ "GUID:3ca842002c51d0a43b73e45298809a13", "GUID:0df5180c0c3a0594fbfa11f83736de9f", "GUID:28964ef7dc9441b6b8671b61a8106690", - "GUID:267f743eefeddd44982a713909628102" + "GUID:267f743eefeddd44982a713909628102", + "GUID:f254b2009e854709a0ef0022a7ca697c" ], "includePlatforms": [ "Editor" diff --git a/Explorer/Explorer.sln.DotSettings b/Explorer/Explorer.sln.DotSettings index 7ba8e7a8064..9a419e73876 100644 --- a/Explorer/Explorer.sln.DotSettings +++ b/Explorer/Explorer.sln.DotSettings @@ -508,6 +508,7 @@ OTP PB PBR + POV SFX SO UISFX diff --git a/docs/README.md b/docs/README.md index ff13159a5d7..c837410b3de 100644 --- a/docs/README.md +++ b/docs/README.md @@ -73,6 +73,7 @@ Welcome to the official documentation for Unity Explorer — the Decentraland cl - **[Debug Container & Widgets](debug-container-and-widgets.md)** — Runtime debug panel architecture, widget builder API, bindings, and integration patterns - **[Testing Guide](testing-guide.md)** — UnitySystemTestBase, ECS test utilities, mocking, EditMode/PlayMode, async test patterns - **[Automation Testing](automation-testing.md)** — AltTester SDK setup, writing UI automation tests, running against instrumented builds and in-Editor, triggering visual regression on PRs via `/visual-tests` +- **[MCP Automation](mcp-automation.md)** — Embedded MCP server for coding agents: screenshots, player/scene state, scene logs, and player control via `--mcp` - **[Connect to Local Scene](how-to-connect-to-a-local-scene.md)** — Running and connecting to local SDK7 scenes - **[Master of Bots](master-of-bots.md)** — Simulating multiple bot users for load testing - **[Override Debug Log Matrix](override-debug-log-matrix.md)** — Runtime log severity overrides diff --git a/docs/app-arguments.md b/docs/app-arguments.md index 7f8f6938688..53f50741d7a 100644 --- a/docs/app-arguments.md +++ b/docs/app-arguments.md @@ -326,6 +326,27 @@ decentraland://?force-open-backpack=true --- +### `mcp` +**Description:** Starts the embedded MCP (Model Context Protocol) server on `http://127.0.0.1:8123/unity-explorer-mcp` so coding agents can observe and drive the client (screenshots, player/scene state, scene logs, teleport/movement, chat commands). The listener binds to localhost only and rejects non-localhost browser Origins. See [MCP Automation](mcp-automation.md). + +**Usage:** +```bash +--mcp +``` + +--- + +### `mcp-port` +**Type:** String (integer port, 1024–65535) +**Description:** Starts the embedded MCP server on a specific port (implies `mcp`). Use distinct ports when running multiple instances via `--multi-instance`. + +**Usage:** +```bash +--mcp-port 8124 +``` + +--- + ### `launch-cdp-monitor-on-start` **Type:** Boolean **Description:** Launches the Chrome DevTools Protocol (CDP) monitor on application start. Enables remote debugging capabilities. diff --git a/docs/mcp-automation.md b/docs/mcp-automation.md new file mode 100644 index 00000000000..7c64bfabe2d --- /dev/null +++ b/docs/mcp-automation.md @@ -0,0 +1,127 @@ +# MCP Automation Server + +The Explorer can host an embedded [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) server so coding agents (e.g. Claude Code) can **see** the running client (screenshots, player/scene state, scene console logs) and **control** it (teleport, move, walk, look, chat commands, scene reload) — closing the edit → reload → verify loop for SDK7 scene development without a human in the middle. + +The server is compiled into all builds but stays dormant unless explicitly enabled at launch. + +--- + +## Enabling + +| Flag | Effect | +|---|---| +| `--mcp` | Starts the MCP server on the default port **8123** | +| `--mcp-port ` | Starts the MCP server on a specific port (implies `--mcp`) | + +The flag is accepted from the command line or a deep link. The endpoint is `http://127.0.0.1:/unity-explorer-mcp`. + +```bash +# macOS +open Decentraland.app --args --mcp + +# Windows +Decentraland.exe --mcp-port 8124 +``` + +In the Unity Editor, add `--mcp` to `Main Scene Loader → Debug Settings → App Parameters`. + +From a scene folder, `@dcl/sdk-commands` can enable it at launch: `npm run start -- --mcp` (optionally `--mcp-port `) forwards both flags into the deep link that auto-launches the installed client. Any extra Explorer params can follow a second standalone `--` (`npm run start -- --mcp -- --windowed-mode --resolution 1280x720`; npm consumes the first `--`). + +## Security model + +- The listener binds to **127.0.0.1 only** — it is never reachable from the network. +- Browser-originated requests are rejected unless their `Origin` is localhost (defense against drive-by pages and DNS rebinding). Requests without an `Origin` header (CLI clients) are allowed. +- The server only exists while the process runs with the flag; there is no persistence and no authentication token in v1. + +## Connecting a coding agent + +```bash +claude mcp add --transport http --scope user explorer http://127.0.0.1:8123/unity-explorer-mcp +``` + +Smoke test without an agent: + +```bash +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}' + +curl -s -X POST http://127.0.0.1:8123/unity-explorer-mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' +``` + +## Tool catalog + +The tables below are a human-readable overview. The authoritative argument contract — exact types, allowed values, defaults — is what the server itself reports via `tools/list`; agents get it fresh at every handshake and should rely on it, not on this page. + +### Seeing + +| Tool | Arguments | Returns | +|---|---|---| +| `screenshot` | `maxWidth?` (default 1280), `quality?`, `worldOnly?` (exclude UI; post-processing still applied) | Downscaled image of the current view (UI included by default) + caption | +| `get_player_state` | — | Player position/rotation/parcel/velocity/grounded + camera position/rotation/mode + wallet address | +| `get_scene_state` | — | Current parcel, scene name/state (incl. `JavaScriptError`/`EcsError`), readiness, loading stage | +| `get_scene_logs` | `limit?`, `severity?`, `sinceSeq?` | Scene JS console output with monotonic sequence numbers for incremental polling | +| `list_scene_entities` | `limit?` | Entity ids of the current scene's ECS world | +| `get_entity_details` | `entityId` | All components of one scene entity | + +### Controlling + +| Tool | Arguments | Effect | +|---|---|---| +| `teleport` | `x`, `y`, `waitForReady?`, `timeoutSec?` | `/goto x,y` through the regular pipeline, waits for scene readiness | +| `move_to` | `x`, `y`, `z`, `lookAt{X,Y,Z}?`, `durationSec?` | Instant or smooth move to a world position (16 m per parcel) | +| `walk` | `directionX`, `directionY`, `seconds?`, `kind?`, `jump?` | Holds camera-relative movement through the real locomotion pipeline (collisions apply) | +| `look_at` | `x`, `y`, `z` | Rotates the camera to a world point (aim before a screenshot) | +| `set_camera_mode` | `mode` | Switches the camera mode like the user hotkey; refuses (with the reason) while a scene locks the camera — `CameraModeArea`, scene virtual camera, or photo camera. `get_player_state` → `camera.modeChangeAllowed` reports the lock state in advance | +| `set_camera_pose` | `x`,`y`,`z`, `lookAt{X,Y,Z}?`, `fov?`, `timeoutSec?` | Places the free camera at an absolute world position, optionally aiming it and setting FOV. Auto-enters free mode (same locks as `set_camera_mode`), waits for the blend to settle (`settled` in the result), and returns the actual pose. The camera stays put while the player moves; restore with `set_camera_mode` | +| `send_chat` | `message` | Sends to Nearby chat; `/commands` run through the chat command pipeline | +| `reload_scene` | `timeoutSec?` | Reloads the current scene (motion + skybox frozen during reload) | +| `trigger_emote` | `urn` or `stop: true`, `loop?` | Plays or stops an avatar emote | +| `click_entity` | `entityId` and/or `x`,`y`,`z` aim point, `button?`, `eventType?`, `timeoutSec?` | Presses a pointer button on a scene entity exactly like a real click: a camera-origin raycast validates the aim (occluders and the entity's `maxDistance` apply), then the entity's pointer-event intent is filled so the scene receives an identical `PBPointerEventsResult`. `click` sends down + up on consecutive scene ticks. Returns `hit`, hover text, hit point/distance, or the blocking entity | + +## Structured output + +`get_player_state`, `get_scene_state` and `list_scene_entities` also return `structuredContent` mirroring their text payload and declare a matching `outputSchema` in `tools/list` (MCP 2025-06-18). This is done **only as an example on the read-only state tools that benefit from it now** — every other tool returns text content only. A tool opts in by overriding `McpTool.OutputSchema` (default `null`); the same `McpJsonSchema` builder produces the schema. + +## The scene-iteration loop + +1. Serve the scene and launch the Explorer in one step: `npm run start -- --mcp` in the scene folder (serves at `http://127.0.0.1:8000`, auto-launches the installed client against it with the MCP server on, and hot-reloads on file changes). +2. To use a specific Explorer build instead, serve with `npm run start -- --no-client` and launch manually: + +```bash +open Decentraland.app --args \ + --realm http://127.0.0.1:8000 --local-scene true --position 0,0 \ + --debug --skip-auth-screen --skip-version-check true \ + --mcp --windowed-mode --resolution 1280x720 +``` + +Optional determinism flags for stable screenshots: `--disable-hud`, `--skybox-time-enabled false`, `--landscape-terrain-enabled false`, `--skip-minimum-specs-screen`. + +3. The agent then loops: edit scene TypeScript → LSD hot reload applies it (or call `reload_scene`) → `get_scene_state` until ready → `screenshot` + `get_scene_logs` → verify → repeat. + +Once loading completes, the server announces its address in the scene debug console (available with local scene development or `--scene-console`): `MCP server listening on http://127.0.0.1:8123/unity-explorer-mcp`. A startup failure (port in use) is announced there as an error instead. The same line lands in the `get_scene_logs` buffer, so agents can confirm the server from inside the loop. + +A user-invokable Claude Code skill wrapping this loop lives at `.claude/skills/mcp-scene-iteration/` (invoke with `/mcp-scene-iteration`). + +## Troubleshooting + +- **Port already in use** — the server logs an `MCP` category error and stays inert; relaunch with a different `--mcp-port`. Multiple Explorer instances (`--multi-instance`) each need their own port. To confirm which process answers on a port, check `serverInfo.pid` in the `initialize` response and the `address` field of `get_player_state`. +- **HTTP 403** — the request carried a non-localhost `Origin` header; MCP clients and curl don't send one. +- **Server won't start on Windows** — `HttpListener` may require a URL ACL depending on machine policy: `netsh http add urlacl url=http://127.0.0.1:8123/unity-explorer-mcp/ user=Everyone` (elevated prompt), then relaunch. +- **Verbose logs** — enabling the server registers a scene-console log handler, which turns on unconditional verbose logging for the session (same behavior as `--scene-console`). +- **Scene entity dumps** — `list_scene_entities`/`get_entity_details` read the scene world without acquiring its sync lock (same as the existing `WorldInfoTool` debug tooling); treat results as a diagnostic snapshot. +- **`click_entity` returns `hit:false` with `blockedBy*`** — another collider sits on the camera→target line; `move_to`/`look_at` to a clear vantage and retry. If the reason is "out of range", close within the entity's `maxDistance` (default 10 m) first. Entities whose collider sits away from the pivot (GLTF meshes) may need an explicit `x/y/z` aim point. + +## Implementation map + +- `Explorer/Assets/DCL/McpServer/` — feature root, its own `DCL.McpServer` assembly. Two folders are folded into other assemblies via `.asmref` so they can reach code that assembly doesn't reference: + - `Core/` — protocol, transport and tool contract: `McpHttpServer` (`HttpListener` server + Origin validation), `McpJsonRpcDispatcher` (JSON-RPC 2.0 routing; `PROTOCOL_VERSION` `2025-06-18`), `McpTool` (abstract tool base), `McpToolsRegistry`, `McpToolResult`, `McpToolAnnotations` (behaviour hints), `McpJsonSchema` (typed schema builder). + - `Tools/` — one class per tool (16). + - `Components/` — ECS components for the input-driving tools: `McpMovementOverride`, `McpPointerEventIntent`. + - `Systems/` — **folded into `DCL.Plugins`** via `.asmref`: `McpServerPlugin` (builds the registry and hosts the server in `InjectToWorld`), `McpInputOverrideSystem` (held movement), `McpPointerEventSystem` (synthetic pointer press/release delivery; `ClickEntityTool` composes a click from two intents). + - `Utils/` — `SceneLogBuffer`, `JObjectExtensions`. + - `Tests/` — EditMode tests **folded into `DCL.EditMode.Tests`** via `.asmref`: dispatcher / registry / result routing and the pointer-click system. +- Gating: `FeatureId.MCP_SERVER` in `FeaturesRegistry` (resolved as `appArgs.HasFlag(MCP) || appArgs.HasFlag(MCP_PORT)`); `DynamicWorldContainer.CreateAsync` reads `FeaturesRegistry.Instance.IsEnabled(FeatureId.MCP_SERVER)` and adds `McpServerPlugin`. +- Flags: `AppArgsFlags.MCP` / `AppArgsFlags.MCP_PORT`; log category: `ReportCategory.MCP`.