Skip to content

UTCP_AGENT.md

Latest commit

 

History

History
401 lines (294 loc) · 9.39 KB

UTCP_AGENT.md

File metadata and controls

401 lines (294 loc) · 9.39 KB
Raw

UTCP Expert LLM Agent

🤖 Overview

The UTCP Documentation MCP Server now includes an OpenAI-powered expert agent that answers questions about UTCP with deep knowledge of the entire specification.

How It Works

User asks Cursor about UTCP
    ↓
Cursor calls ask_utcp_expert tool on MCP server  
    ↓
MCP Server's LLM Agent:
  1. Searches UTCP docs for relevant context
  2. Calls OpenAI API with docs + question
  3. Returns expert answer
    ↓
User gets intelligent response

🆕 New Tool: ask_utcp_expert

An AI agent with deep UTCP knowledge that can answer any question about the protocol.

Usage

{
  "tool": "ask_utcp_expert",
  "arguments": {
    "question": "How do I authenticate with bearer tokens in HTTP?"
  }
}

Example Questions

Explanations:

  • "What is body_template and when should I use it?"
  • "Explain the difference between SSE and streamable_http protocols"
  • "What's the purpose of call_template_type?"

How-to:

  • "How do I add OAuth2 authentication to my UTCP manual?"
  • "How do I reference previous CLI command outputs?"
  • "How do I handle large streaming responses?"

Examples:

  • "Show me a complete HTTP tool example with authentication"
  • "Give me an example of a CLI tool that chains multiple commands"
  • "What does a working SSE tool look like?"

Troubleshooting:

  • "Why am I getting 401 errors with my bearer token?"
  • "My tool name is failing validation, what's wrong?"
  • "How do I fix 'body_template required' error?"

Response Format

The agent returns:

  • Detailed answer - Explained clearly with context
  • Code examples - When relevant, includes JSON examples
  • Sources - Lists which UTCP docs were used
  • Best practices - Suggests proper approaches

Example Response:

# Bearer Token Authentication

Bearer tokens are commonly used for OAuth2 and JWT authentication in HTTP APIs.

## Configuration

Add this to your tool's auth section:

```json
{
  "auth": {
    "auth_type": "bearer",
    "token": "${BEARER_TOKEN}"
  }
}

Usage in Headers

The bearer token is automatically added to the Authorization header:

Authorization: Bearer your_token_here

Best Practices

  1. Store tokens in environment variables
  2. Never commit tokens to code
  3. Use HTTPS only for bearer auth
  4. Consider token expiry and refresh

📚 Sources

  1. HTTP Protocol (protocols) - Relevance: 95
  2. Authentication Guide (guides) - Relevance: 88
  3. for-tool-providers (guides) - Relevance: 76

## 🔧 Configuration

### Environment Variable

Set your OpenAI API key:

```bash
export OPENAI_API_KEY="sk-your-api-key-here"

In .env file:

# .env
OPENAI_API_KEY=sk-your-api-key-here

In Cursor/Claude Desktop config:

{
  "mcpServers": {
    "utcp-docs": {
      "command": "node",
      "args": ["/path/to/utpc-docs-mcp-server/dist/index.js"],
      "env": {
        "OPENAI_API_KEY": "sk-your-api-key-here"
      }
    }
  }
}

🧠 How The Agent Works

1. Context Retrieval

When you ask a question, the agent:

  • Searches the UTCP documentation for relevant sections
  • Retrieves top 5-10 most relevant docs
  • Includes up to 2000 characters from each doc

2. Example Detection

If your question needs examples, it automatically includes:

  • Protocol-specific examples (HTTP, CLI, SSE, etc.)
  • Authentication examples (if asking about auth)
  • Minimal working examples

3. LLM Generation

The agent calls OpenAI's GPT-4 Turbo with:

  • System prompt - Makes it a UTCP expert
  • Documentation context - Relevant UTCP docs
  • Your question - What you asked

4. Source Attribution

Every response includes sources showing which docs were used.

🎯 Architecture

Clean & Simple

class UtcpAgent {
  constructor(
    docService: DocumentationService,
    generator: UtcpGenerator,
    apiKey?: string
  )

  async ask(question: string): Promise<{
    answer: string;
    sources: Array<{
      title: string;
      section: string;
      relevance: number;
    }>;
  }>
}

No Complexity

  • ✅ No session management (stateless)
  • ✅ No complex intent classification
  • ✅ No conversation state tracking
  • ✅ Simple: question in → answer out

Smart Context Building

The agent intelligently builds context by:

  1. Searching docs for your question
  2. Detecting if examples are needed
  3. Adding protocol-specific examples
  4. Adding auth examples if relevant

💡 Why This Works

For Users

  1. Natural Q&A - Ask like talking to an expert
  2. Accurate Answers - Grounded in actual UTCP docs
  3. With Examples - Gets relevant code automatically
  4. Source References - See which docs were used

For MCP Server

  1. Leverages existing search - Uses DocumentationService
  2. No duplication - Doesn't reimplement anything
  3. Extensible - Easy to improve prompts
  4. Cost-effective - Only calls API when needed

🚀 Benefits Over Simple Search

Feature search_utcp_docs ask_utcp_expert
Input Keywords Natural language question
Understanding Keyword matching Semantic understanding
Response Raw doc excerpts Explained answer
Examples Must search separately Included automatically
Context Single query Multiple docs synthesized
Guidance None Best practices included

🎨 Example Workflows

Learning UTCP

// Question 1
ask_utcp_expert({
  question: "What are the main protocol types in UTCP?"
})
// Gets: Overview of HTTP, CLI, MCP, SSE, Streamable HTTP

// Question 2
ask_utcp_expert({
  question: "Show me a complete HTTP tool example with API key auth"
})
// Gets: Full example + explanation + best practices

Building a Manual

// Question 1
ask_utcp_expert({
  question: "I need to create a tool for GitHub's API. What protocol should I use?"
})
// Gets: Recommendation for HTTP protocol + reasoning

// Question 2  
ask_utcp_expert({
  question: "How do I structure the GitHub API tool with bearer token auth?"
})
// Gets: Complete example + field explanations

// Then use the actual tools
generate_utcp_manual({...})
validate_utcp_manual({...})

Debugging

ask_utcp_expert({
  question: "My tool name 'GetData' is failing validation. What's wrong?"
})
// Gets: Explanation of snake_case requirement + how to fix

🔒 Security & Privacy

API Key

  • Stored in environment variables
  • Never logged or exposed
  • Required for agent to work

Data

  • Your questions are sent to OpenAI
  • UTCP docs are sent as context
  • No personal data unless you include it in questions

Costs

  • Uses GPT-4 Turbo (~$0.01 per 1K tokens)
  • Typical question costs $0.02-0.05
  • Monitor usage in OpenAI dashboard

🎯 Tips for Best Results

Ask Clear Questions

✅ Good:

  • "How do I add OAuth2 authentication to an HTTP tool?"
  • "Show me an example of a CLI tool with multiple commands"
  • "What's the difference between body_template and body_field?"

❌ Less Good:

  • "auth?" (too vague)
  • "how work" (unclear)
  • "example" (need more context)

Request Examples

Add "show me" or "example" to get code:

  • "Show me an SSE protocol example"
  • "Give me an example of bearer auth"
  • "Example of CLI tool with working directory"

Build on Answers

The agent has full UTCP context, so you can:

  • Ask follow-up questions
  • Request deeper explanations
  • Get alternative approaches

📊 Comparison to Other Approaches

vs. Static Documentation

Static Docs LLM Agent
Must search & read Ask directly
Find relevant sections manually Automatic retrieval
Piece together info Synthesized answer
No examples Examples included

vs. RAG Without LLM

Simple RAG LLM Agent
Returns chunks Returns answers
User interprets LLM explains
Context limited Context synthesized
No reasoning Can reason & suggest

🔮 Future Enhancements

Possible improvements:

  • Support for other LLM providers (Anthropic, local models)
  • Fine-tuning on UTCP-specific Q&A pairs
  • Caching frequent questions
  • Multi-turn conversation support
  • Code execution to validate examples

🐛 Troubleshooting

"OpenAI API key not configured"

Solution: Set the OPENAI_API_KEY environment variable

export OPENAI_API_KEY="sk-..."

Agent responses are slow

Reason: GPT-4 Turbo can take 5-10 seconds for detailed answers

Solutions:

  • Use GPT-3.5-turbo for faster (but less accurate) responses
  • Reduce context size in agent configuration

Agent gives wrong answers

Reason: Question might be too vague or docs not comprehensive enough

Solutions:

  • Ask more specific questions
  • Include more context in your question
  • Use the simpler search tools for quick lookups

📝 Summary

The UTCP Expert Agent transforms the MCP server from a documentation search tool into an intelligent assistant that:

✅ Understands natural language questions
✅ Retrieves relevant UTCP documentation
✅ Synthesizes comprehensive answers
✅ Includes code examples automatically
✅ Explains concepts clearly
✅ Suggests best practices
✅ Sources its answers transparently

All powered by OpenAI, grounded in actual UTCP docs, and accessible through a simple MCP tool!

Try it: ask_utcp_expert({ question: "How does the UTCP agent work?" })