autonomous lungs segmentation tool call

[qchapp]

This pull request introduces major improvements to the agent tool system, UI workflow, and tool registry architecture, while removing autonomous tool execution for enhanced security and user control. Notably, it adds a centralized tool registry, a new 3D lungs segmentation tool, improved download/display separation for tool results, and a button-based approval workflow. The agent now only recommends tools, requiring explicit user approval for execution, and the UI has been redesigned for clarity and extensibility.

Agent Tool System and Registry Architecture:

• Removed autonomous tool execution (run_example), enforcing explicit user approval for all tool executions; agent now only recommends tools, aligning with a consistent security and UX model. ([[1]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR7-R9), [[2]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-8fbbb001f5cf78a68114700715b3ce8a45876768c65bd8cba85165db4c8bb2c3L16-L21), [[3]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-8fbbb001f5cf78a68114700715b3ce8a45876768c65bd8cba85165db4c8bb2c3L219-L251), [[4]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-8fbbb001f5cf78a68114700715b3ce8a45876768c65bd8cba85165db4c8bb2c3L390), [[5]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-8fbbb001f5cf78a68114700715b3ce8a45876768c65bd8cba85165db4c8bb2c3L406))
• Introduced a centralized tool registry system (agent/tools/mcp/registry.py) with declarative configuration, dynamic tool lookup, catalog name mapping, and lazy loading to avoid heavy dependency imports at startup. ([[1]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR40-R63), [[2]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-c147d0280cf73806958f602a78e5a58ea03fe39ab3e1ee5f49cf16253c246b2dR1-R36), [[3]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-0767b1334225f80dfdd3616abd17fa846d2704241dbc9149c7bbe4c2ead8783eR1-R51))
• Established an MCP tools subpackage (agent/tools/mcp/) and base tool models (agent/tools/mcp/base.py) for standardized input/output schemas and consistent UI integration. ([[1]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR40-R63), [[2]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-0767b1334225f80dfdd3616abd17fa846d2704241dbc9149c7bbe4c2ead8783eR1-R51), [[3]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-7b430ae86ee6ff5867d7d0c178f6f61e1d233faa08c67649bd6c167d0cdceeb9R1-R90))

New Features and Tool Enhancements:

• Added the 3D Lungs Segmentation Tool with robust file handling, supporting DICOM, NIfTI, and TIFF stack inputs, and self-registration in the tool registry. ([CHANGELOG.mdR40-R63](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR40-R63))
• Enhanced tool output schemas to separate display previews (result_preview) from downloadable originals (result_origin), improving compatibility and user access to results. ([[1]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR90-R114), [[2]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-7b430ae86ee6ff5867d7d0c178f6f61e1d233faa08c67649bd6c167d0cdceeb9R1-R90))
• Introduced tool usage analytics and downloadable results section in the UI for real-time transparency and easy access to outputs. ([CHANGELOG.mdR40-R63](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR40-R63))

UI/UX Improvements:

• Replaced text-based tool approval with inline button-based approval, improving clarity and preventing accidental execution; approval button now appears dynamically in chat flow within a styled group box. ([CHANGELOG.mdR90-R114](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR90-R114))
• Redesigned chat output structure and event handler yields to consistently support new approval and download components, with all handlers now returning 9 values. ([[1]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR90-R114), [[2]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR132-R135))
• Moved approval button inline between chatbot and downloads section, using registry helpers for dynamic labels and icons. ([CHANGELOG.mdR90-R114](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR90-R114))

Compatibility and Maintenance:

• Fixed Gradio component compatibility by switching from gr.Box to gr.Group, and ensured consistent output counts across UI handlers. ([CHANGELOG.mdR132-R135](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR132-R135))
• Removed duplicate function definitions and legacy text-based approval logic to prevent errors and ambiguity. ([CHANGELOG.mdR132-R135](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR132-R135))
• Updated default model configuration in config.yaml to use EPFL inference server and new model name. ([config.yamlL5-R10](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-d8d0422389f03d783e32e627250fe29834bd09c6361640d1ff00661dd6820034L5-R10))

Agent Output and Analytics:

• Added token usage statistics to agent output for improved tracking and analytics. ([[1]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-dfbed62c9d71f2b991784da935a91fa3362a5f410c10ef5b898dcf1c6b07a1afR14-R22), [[2]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-dfbed62c9d71f2b991784da935a91fa3362a5f410c10ef5b898dcf1c6b07a1afR32-R38), [[3]](https://github.com/Imaging-Plaza/ai-agent/pull/22/files#diff-8fbbb001f5cf78a68114700715b3ce8a45876768c65bd8cba85165db4c8bb2c3L493-R474))

---

References:
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]

[Copilot]

Pull request overview

This pull request introduces a comprehensive overhaul of the agent tool system, transitioning from autonomous tool execution to a user-controlled approval workflow. The changes establish a centralized tool registry architecture, add a new 3D lungs segmentation tool, implement button-based approval mechanisms, and enhance the UI with analytics and downloadable results.

Changes:

• Removed autonomous run_example tool from agent, enforcing explicit user approval for all tool executions via new approval button workflow
• Introduced centralized tool registry system (agent/tools/mcp/registry.py) with declarative ToolConfig, catalog name mapping, and dynamic tool lookup
• Added 3D Lungs Segmentation tool with robust file materialization handling multiple Gradio output formats (FileData dict, URL string, local path, server path)
• Enhanced UI with inline approval buttons, downloadable results section, performance stats display, and tool usage analytics (bar chart + timeline)
• Updated output schemas to separate display previews (result_preview) from downloadable originals (result_origin) for better compatibility

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/ai_agent/agent/tools/mcp/registry.py	New centralized tool registry with ToolConfig dataclass, dynamic lookup, and catalog name mapping
src/ai_agent/agent/tools/mcp/lungs_segmentation_tool.py	New 3D lungs segmentation tool with robust file materialization from HuggingFace Space
src/ai_agent/agent/tools/mcp/base.py	Base Pydantic models (BaseToolInput, BaseToolOutput, ImageToolInput) for standardized tool schemas
src/ai_agent/agent/tools/mcp/__init__.py	MCP tools package exports and lazy loading mechanism
src/ai_agent/agent/tools/__init__.py	Updated tools package with ensure_tools_registered() function
src/ai_agent/ui/state.py	Extended ChatState with tool approval fields (pending_tool_approval, pending_tool_params, agent_result) and ChatMessage with stats rendering
src/ai_agent/ui/handlers.py	Added generic execute_tool_with_approval() using registry for dynamic tool dispatch; updated imports for MCP registry
src/ai_agent/ui/components.py	Added approval button group, download files section, handle_tool_approval() handler, and extended all outputs to 9 values
src/ai_agent/ui/app.py	Added tool registration call via ensure_tools_registered() at startup
src/ai_agent/agent/models.py	Added UsageStats model and integrated into AgentToolSelection
src/ai_agent/agent/agent.py	Removed run_example tool, added usage stats extraction from RunResult
src/ai_agent/generator/prompts.py	Removed run_example from available tools documentation
config.yaml	Updated default model to EPFL inference server endpoint with placeholder model name
CHANGELOG.md	Comprehensive documentation of all changes

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 6 comments.

[qchapp]

Hi @caviri, could you please review this PR? Thanks!

[caviri]

Hi @qchapp, looks good to me. You can merge it to develop :)