feat(agent): deferred MCP tool loading via tool_search

[etano]

What

Adds opt-in deferred tool loading for MCP tools via a built-in tool_search meta-tool, so setups with many MCP servers don't pay the full tool-schema context cost on every turn.

Why

Every connected MCP server's tool schemas are loaded into context on every request. A handful of servers can add tens of thousands of tokens before any work happens, and tool-selection accuracy degrades once the toolset grows past ~30–50 tools. Deferring MCP tools behind a search keeps the per-turn context small and the visible toolset focused.

How it works

• New options.tool_search config: { "enabled": bool, "threshold": int (default 20) }.
• When enabled and the number of MCP tools exceeds threshold, buildTools partitions tools into an always-loaded core (built-ins) and a deferred catalog (MCP tools). The model sees only core + a tool_search tool.
• The model calls tool_search with keywords; matching catalog tools are activated and pushed onto the running agent via SetTools. Because tools are recomputed each step in PrepareStep, they become callable on the next turn — no restart.
• Activated tools persist for the conversation and reset when the MCP catalog changes.

Notes / scope

• Opt-in (default off) — no behavior change unless configured.
• Provider-agnostic — implemented client-side (a normal tool + the existing dynamic tool list), so it works with any model, not just Anthropic.
• Top-level agent only — sub-agents are unaffected.
• PreToolUse hooks are preserved on activated tools (both partitions are hook-wrapped).
• Ranking is a dependency-free case-insensitive token-overlap over tool name + description (name matches weighted higher). Straightforward to swap for BM25/embeddings later.

Testing

• Unit tests for ranking, activation (cap + dedupe), effective-set composition, catalog-change reset, and tokenization.
• go build ./... clean, schema.json regenerated; verified end-to-end against a live setup with 40+ MCP tools — schemas deferred out of context, model discovers and calls tools on demand.

Config example

{
  "options": {
    "tool_search": { "enabled": true, "threshold": 20 }
  }
}

[charmcli]

Thank you for your submission. We really appreciate it! Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request comment same as the below format.

---

I have read the Contributor License Agreement (CLA) and hereby sign the CLA.

---

Ethan Brown seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}