Feat/swarm mcp tools

[shadowinlife]

Summary

• Enables SWARM workers inside run_swarm to use operator-approved external MCP tools.
• Preserves the trust boundary: callers cannot inject MCP server URLs, commands, env vars, or allowlist overrides.
• Adds M1-M6 regression coverage and docs for the SWARM external MCP tools rollout.

Why

SWARM analyst workers could previously only use local Vibe-Trading tools, even when Vibe-Trading itself was exposed through MCP. This prevented enterprise/internal workflows from letting analysts query approved knowledge bases, data lakes, compliance tools, or other MCP-backed data sources.

This PR adds server-side, operator-controlled MCP tool access for SWARM workers while keeping caller-provided variables as template data only.

Changes

• Threads optional AgentConfig through SwarmRuntime into run_worker.
• Adds build_swarm_registry(...) to merge local tools with configured remote MCP wrappers, then filter by each agent's tools: whitelist.
• Adds SWARM MCP config resolution:
  • VIBE_TRADING_SWARM_AGENT_CONFIG
  • ~/.vibe-trading/swarm-agent.json
  • fallback to ~/.vibe-trading/agent.json
  • empty config for legacy local-only behavior
• Wires SWARM MCP config into MCP server, API server, CLI live swarm runner, and in-process SwarmTool.
• Adds remote MCP metadata to tool_call / tool_result events via server and remote_tool, while keeping sensitive args redacted.
• Adds trust-model tests proving run_swarm does not expose MCP config injection fields and does not derive MCP config from caller variables.
• Adds roadmap/TDD docs and README section for SWARM external MCP tools.

Test Plan

• Existing tests pass (pytest --ignore=agent/tests/e2e_backtest --tb=short -q)
• New tests added (if applicable)
• Tested manually (describe below)

Checklist

• No changes to protected areas (src/agent/, src/session/, src/providers/) without prior discussion
• No hardcoded values (API keys, file paths, magic numbers)
• Code follows CONTRIBUTING.md guidelines
• Documentation updated (if user-facing change)

[Copilot]

Pull request overview

This PR extends the SWARM execution path (run_swarm workers) to use operator-approved external MCP tools while preserving the trust boundary (callers cannot inject MCP server config via variables). It threads an optional AgentConfig through SwarmRuntime into run_worker, builds a swarm-specific tool registry that merges local + remote MCP tools, and adds extensive regression coverage plus documentation for the rollout.

Changes:

• Add SWARM-side MCP config resolution (VIBE_TRADING_SWARM_AGENT_CONFIG → swarm-agent.json → agent.json → empty config) and wire it into MCP server, API server, CLI, and in-process SwarmTool.
• Introduce build_swarm_registry(...) and update workers to use it; emit server/remote_tool metadata for remote tool calls in tool_call/tool_result events.
• Add M1–M6 regression tests (plumbing, registry assembly, boot wiring, e2e, trust model) and new docs/README section.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
README.md	Document SWARM external MCP tools and link to roadmap/TDD docs.
docs/2026-05-25_swarm_mcp_tools_tdd.md	Add TDD matrix with requirements/scenarios/tests and status tracking.
docs/2026-05-25_swarm_mcp_tools_roadmap.md	Add/mark implemented roadmap for SWARM external MCP tools and trust model.
agent/tests/test_swarm_m5_trust_model.py	Add trust-boundary regression tests for run_swarm schema and variable handling.
agent/tests/test_swarm_m4_e2e.py	Add end-to-end worker tests covering remote MCP calls, metadata, redaction, and failure handling.
agent/tests/test_swarm_m3_boot_wiring.py	Add tests for SWARM agent config path resolution and loader behavior.
agent/tests/test_swarm_m2_registry_assembly.py	Add tests for merging/filtering local + remote MCP tools via build_swarm_registry.
agent/tests/test_swarm_m1_config_plumbing.py	Add tests ensuring agent_config plumbing and lazy behavior at construction time.
agent/src/tools/swarm_tool.py	Load SWARM agent config for in-process swarm tool execution.
agent/src/tools/init.py	Add build_swarm_registry and refactor filtered registry projection helper.
agent/src/swarm/worker.py	Use build_swarm_registry and emit remote MCP metadata in events.
agent/src/swarm/runtime.py	Thread optional agent_config through runtime into worker execution.
agent/src/config/loader.py	Implement SWARM config resolution + loader helper.
agent/src/config/init.py	Export load_swarm_agent_config.
agent/mcp_server.py	Load SWARM agent config before constructing SwarmRuntime in MCP run_swarm.
agent/cli/_legacy.py	Load SWARM agent config before constructing runtime in CLI live runner.
agent/api_server.py	Load SWARM agent config before constructing cached runtime in API server.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[warren618]

Thanks for the quick update. I don't think this is mergeable yet.

The new pruning path in build_swarm_registry() breaks the existing MCP server-name collision handling. _prune_agent_config_for_swarm_tools() computes the local server-name segments from the full config, but then returns a new AgentConfig containing only the selected servers. build_registry() recomputes resolve_mcp_server_tool_name_segments() on that pruned config, so a colliding server such as foo-bar / foo_bar loses its hash suffix after pruning.

Example: the full config may expose mcp_foo_bar_db7329d5_ping, but after pruning build_registry() rebuilds it as mcp_foo_bar_ping, so the worker whitelist no longer matches and the tool is dropped as unavailable.

Could you preserve the original resolved local server-name mapping when pruning, or skip pruning for collision groups, and add a regression test with two colliding server names?

[shadowinlife]

@warren618 good catch — fixed in c4e107f.

The fix preserves the original resolved server-name mapping rather than skipping pruning:

1. _prune_agent_config_for_swarm_tools() now resolves local_server_names against the full config (so foo-bar / foo_bar collisions get their hash suffixes assigned), then returns both the pruned AgentConfig and the subset of name segments that belong to the retained servers — as a tuple. (agent/src/tools/__init__.py:235-263)
2. build_registry() gained a private _mcp_server_tool_name_segments parameter. When the swarm path supplies it, build_registry() skips its own re-resolution and uses the caller-supplied mapping — so the disambiguation computed against the full config survives pruning verbatim. (agent/src/tools/__init__.py:74, 137-146)
3. Regression test test_build_swarm_registry_preserves_collision_hash_prefix_after_pruning exercises exactly the foo-bar / foo_bar case you described: constructs both servers, derives the expected suffixed tool name via resolve_mcp_server_tool_name_segments(), requests it through the whitelist, and asserts the registry contains the suffixed name and build_mcp_tool_wrappers was called with the matching local_server_name=. (agent/tests/test_swarm_m2_registry_assembly.py:221-247)

All 27 M-series tests pass locally on c4e107f. PTAL.