Feature/mcp support (#74)

* feat: Implement MCP server integration with navigation and tool execution capabilities

- Added mcp_server.py to define basic math tools and server file retrieval.
- Introduced mcp_servers.json for server configuration.
- Updated poetry.lock to include MCP and related dependencies.
- Enhanced test feature to validate MCP server connection and calculations.
- Implemented MCP Navigation Agent for executing tools and managing resources.
- Developed MCP tools for server communication and tool execution, including connection management and error handling.
- Configured environment variables for MCP integration in BaseConfigManager.

* working mcp client

* bug fix

* deps update

* removed unwanted file

* update gitignore

* feat: Enhance MCP support with detailed documentation and configuration

- Added MCP integration section in README.md, including quick start guide and example usage.
- Introduced MCP Integration details in environment_variables.md, outlining configuration options.
- Created MCP_Usage.md to provide comprehensive guidance on connecting Hercules to MCP servers and executing tools.
- Updated simple_hercules.py to ensure MCP agents are properly set up.
- Refactored mcp_tools.py for improved organization and clarity in MCP server communication.

Author: Shahalt1

.gitignore

@@ -26,6 +26,7 @@ share/python-wheels/
 .installed.cfg
 *.egg
 MANIFEST
+mcp_servers.json
 
 # PyInstaller
 #  Usually these files are written by a python script from a template

README.md

@@ -494,6 +494,50 @@ To configure Hercules in detail:
   cp .env-example .env
   ```
 
+## 🔌 MCP Support
+
+Hercules includes an MCP Navigation Agent that can connect to MCP servers over `stdio`, `sse`, and `streamable-http`. It works out of the box with Composio-generated MCP servers.
+
+- Quick start guide: see `docs/MCP_Usage.md`
+- Composio docs: https://docs.composio.dev/docs/mcp-developers
+
+Minimal setup:
+
+1) Create `mcp_servers.json` at the repo root:
+
+```
+{
+  "mcpServers": {
+    "server_name": {
+      "transport": "streamable-http",
+      "url": "https://mcp.composio.dev/composio/server/<SERVER_UUID>/mcp?user_id=<USER_EMAIL>"
+    }
+  }
+}
+```
+
+2) In `.env`:
+
+```
+MCP_ENABLED=true
+MCP_SERVERS=mcp_servers.json
+MCP_TIMEOUT=30
+```
+
+Then, run Hercules as usual; the MCP agent will initialize and expose tools from the configured server.
+
+### Example Testcase (Simple)
+```
+Feature: Read emails from Gmail
+
+  Scenario: Retrieve OTP from Gmail using MCP
+    Given Gmail is configured using MCP
+    When I connect to Gmail
+    And I read the email with OTP
+```
+For more testcases visit [MCP-Docs](/docs/MCP_Usage.md)
+
+---
 - Hercules is capable of running in two configuration forms:
 
   1. **Using single LLM for all work**

docs/MCP_Usage.md

@@ -0,0 +1,114 @@
+# Using MCP with Hercules
+
+This guide explains how to connect Hercules to an MCP server and execute tools, with Composio as a ready-to-use provider.
+
+## Overview
+- Hercules ships with an MCP Navigation Agent and MCP tooling.
+- Supports `stdio`, `sse`, and `streamable-http` transports.
+- Optimized for servers that expose tools but may not expose resources.
+
+## Pre-requisites
+- Hercules installed and running (PyPI, Docker, or source).
+- A valid MCP server URL. For Composio, use the generated server URL which embeds user scoping.
+  - Pattern: `https://mcp.composio.dev/composio/server/<SERVER_UUID>/mcp?user_id=<USER_EMAIL>`
+  - Composio docs: https://docs.composio.dev/docs/mcp-developers
+
+## Configure Hercules
+
+1) Create `mcp_servers.json` in the repository root:
+
+```json
+{
+  "mcpServers": {
+    "server_name": {
+      "transport": "streamable-http",
+      "url": "https://mcp.composio.dev/composio/server/<SERVER_UUID>/mcp?user_id=<USER_EMAIL>"
+    }
+  }
+}
+```
+
+- `server_name`: Logical name used to reference this server in tools.
+- `transport`: Use `streamable-http` for Composio’s HTTP streaming transport. `stdio` and `sse` are also supported if your server provides them.
+- `url`: Your server URL; replace placeholders with your actual values.
+
+2) Enable MCP in your environment (e.g., `.env`):
+
+```
+MCP_ENABLED=true
+MCP_SERVERS=mcp_servers.json
+# Optional timeout in seconds
+MCP_TIMEOUT=30
+```
+
+
+
+Hercules manages client contexts and sessions, and registers MCP tools for the agent to call.
+
+## Transport Notes
+- `streamable-http`: Recommended for Composio; avoids websocket restrictions and supports streaming.
+- `sse`: Use if your server provides an SSE endpoint.
+- `stdio`: Use for local MCP servers launched as subprocesses.
+
+## Troubleshooting
+- “Method not found” during connect
+  - Cause: Server doesn’t implement resource endpoints.
+  - Resolution: Hercules initializes the MCP toolkit with resources disabled by default and tools enabled. Ensure you are on a version with this behavior.
+- `Server not found in connected toolkits`
+  - Ensure `MCP_ENABLED=true`, `MCP_SERVERS` points to `mcp_servers.json`, and `initialize_mcp_connections` has been called.
+- `Connection failed`
+  - Verify the URL (UUID, user email) and that the server is reachable over HTTPS.
+
+## Security Notes
+- Do not commit user emails or tokens. Keep server URLs and environment in private configuration.
+- Composio server URLs handle authentication and tool access based on your server configuration.
+
+## References
+- Composio MCP: https://docs.composio.dev/docs/mcp-developers
+
+## Example Testcase (Simple)
+
+```gherkin
+Feature: Read emails from Gmail
+
+  Scenario: Retrieve OTP from Gmail using MCP
+    Given Gmail is configured using MCP
+    When I connect to Gmail
+    And I read the email with OTP
+```
+
+## Concrete OTP Retrieval Scenario
+
+The following example demonstrates a full flow using MCP tools to search, fetch, and extract an OTP from an email. Replace tool names and fields to match your MCP server’s tool schema.
+
+```gherkin
+Feature: Retrieve OTP via MCP Gmail
+
+  Background:
+    Given MCP is enabled and the "mail" server is configured
+    And I initialize MCP connections
+    And MCP server "mail" is connected
+
+  Scenario: Read OTP from most recent email
+    When I execute MCP tool "GMAIL_FETCH_EMAILS" on "mail" with:
+      """
+      {"query": "subject:(OTP OR verification code) newer_than:15m", "max_results": 5}
+      """
+    Then I store the first result's "id" as "message_id"
+
+    When I execute MCP tool "GMAIL_FETCH_MESSAGE_BY_MESSAGE_ID" on "mail" with:
+      """
+      {"id": "${message_id}"}
+      """
+    Then I read OTP from the email body
+    And I should see an OTP
+```
+
+Notes:
+- Typical tool names on Composio MCP servers:
+  - 'GMAIL_FETCH_EMAILS'
+  - 'GMAIL_FETCH_MESSAGE_BY_MESSAGE_ID'
+  - 'GMAIL_FETCH_MESSAGE_BY_THREAD_ID', 
+- Typical request fields:
+  - For search: query, max_results
+  - For get message: id or message_id

docs/environment_variables.md

@@ -27,6 +27,7 @@ This document provides a comprehensive guide to all environment variables and co
 - [Advanced Configuration](#advanced-configuration)
   - [Cache and Performance](#cache-and-performance)
   - [Portkey Integration](#portkey-integration)
+  - [MCP Integration](#mcp-integration)
   - [Telemetry and Monitoring](#telemetry-and-monitoring)
   - [Additional Features](#additional-features)
 - [Configuration Priority](#configuration-priority)
@@ -335,6 +336,26 @@ LLM Parameters:
   - Default: `30.0`
   - Implementation: Controls how long Portkey waits for a response before timing out
 
+### MCP Integration
+- `MCP_ENABLED`: Enable MCP integration
+  - Values: `true`, `false`
+  - Default: `false`
+  - Implementation: Toggles MCP agent and tooling. When enabled, Hercules can connect to configured MCP servers.
+
+- `MCP_SERVERS`: MCP server configuration
+  - Values: Path to a `.json` file or a JSON string
+  - Example (file path): `mcp_servers.json`
+  - Example (JSON string): `{"mcpServers": {"server_name": {"transport": "streamable-http", "url": "https://mcp.composio.dev/composio/server/<SERVER_UUID>/mcp?user_id=<USER_EMAIL>"}}}`
+  - Implementation: If a path ending with `.json` is provided, Hercules attempts to load it from:
+    - The current working directory
+    - Relative to `testzeus_hercules/`
+    - Relative to `testzeus_hercules/config.py` location
+    Otherwise, it treats the value as a JSON string.
+
+- `MCP_TIMEOUT`: MCP read timeout in seconds
+  - Default: `30`
+  - Implementation: Used to set `ClientSession` read timeouts during MCP interactions.
+
 ### Telemetry and Monitoring
 - `ENABLE_TELEMETRY`: Enable usage telemetry
   - Values: `0`, `1`

mcp_servers.example.json

@@ -0,0 +1,8 @@
+{
+    "mcpServers": {
+        "server_name":{
+            "transport": "streamable-http",
+            "url": "https://mcp.composio.dev/composio/server/<UUID>/mcp?user_id=user123@example.com"
+        }
+    }
+}