Production Research Copilot

A multi-agent research assistant that plans a research task, calls tools (web search, RAG), synthesizes the findings into a structured markdown report, and saves it into a Git repository for long‑term knowledge management.

Built with FastAPI, Groq (OpenAI‑compatible), and a modular agent architecture so you can easily swap models, tools, and vector stores.

---

Why this project exists

Most “AI chatbot” demos are single‑prompt UIs that don’t show how to:

• Decompose a task into multiple steps
• Call external tools (web search, RAG, git) reliably
• Persist artifacts (reports) into real systems
• Wire everything into an HTTP API that could run in production

This project is designed as a portfolio‑grade example of a production‑oriented multi‑agent system aimed at AI Engineer / Agent Developer roles:

• It plans multi‑step research workflows
• It uses specialized agents with clear responsibilities
• It integrates with external tools (web search, vector DB, git)
• It exposes a clean FastAPI endpoint you can deploy anywhere

---

High-level architecture

The system is composed of:

• FastAPI server – HTTP API (/api/research) and health check (/)
• Orchestrator – Coordinates agents and shared state for each request
• Planner agent – Breaks the user query into ordered steps with tools
• Web research agent – Calls a web search tool, summarizes results
• RAG agent – Queries a local vector store built from your own docs
• Writer agent – Synthesizes a structured markdown report
• Git writer tool – Writes the report to disk and (optionally) commits to git
• Config + LLM client – Abstracts the underlying provider (Groq by default)

Architecture diagram (Mermaid)

flowchart LR
    U[User] -->|POST /api/research| API[FastAPI Server]

    API --> ORCH[Orchestrator]

    ORCH --> PL[Planner Agent]
    PL --> ORCH

    ORCH --> WA[Web Research Agent]
    ORCH --> RA[RAG Agent]
    WA --> WS[Web Search Tool]
    RA --> VS[Vector Store]

    WA --> ORCH
    RA --> ORCH

    ORCH --> WR[Writer Agent]
    WR --> ORCH

    ORCH --> GW[Git Writer Tool]
    GW --> RF[Markdown Report MD]

Loading

---

Tech stack

• Backend
  • Python 3.11+
  • FastAPI + Uvicorn
• LLM
  • Groq API via OpenAI‑compatible chat endpoint (e.g. llama-3.3-70b-versatile)
• Agents & tools
  • Custom planner, web, RAG, writer, and orchestrator classes
• Retrieval
  • Chroma (local persistent vector store) or pluggable embedding backend
• Persistence
  • Markdown reports saved under reports/
  • Optional git commits via subprocess
• Config & settings
  • pydantic-settings for .env management
• Containerization
  • Dockerfile for easy deployment (Render, Fly.io, etc.)
• Testing
  • Pytest unit tests for planner and orchestrator

---

Repository structure

production-research-copilot/
  app/
    __init__.py
    config.py            # Settings (Groq, paths, etc.)
    llm_client.py        # LLM abstraction (OpenAI-compatible, Groq by default)
    logging_config.py
    schemas.py           # Pydantic request/response models
    server.py            # FastAPI app

    agents/
      __init__.py
      base.py            # BaseAgent
      planner.py         # PlannerAgent
      web_agent.py       # WebResearchAgent
      rag_agent.py       # RAGAgent
      writer_agent.py    # WriterAgent
      orchestrator.py    # Orchestrator

    tools/
      __init__.py
      web_search.py      # WebSearchTool (stub or real search API)
      rag_retriever.py   # RAGRetriever (Chroma)
      git_writer.py      # GitWriter (writes & commits reports)

  data/
    docs/                # Your documents for RAG
    vector_store/        # Chroma persistent data

  reports/               # Generated markdown reports

  tests/
    test_planner.py
    test_orchestrator.py

  .env.example
  config.example.yaml
  .gitignore
  requirements.txt
  Dockerfile
  README.md

---

How the agent workflow works

1. User sends a query to /api/research:

   • e.g. “Best open‑source frameworks for building production AI agents in 2026.”

2. Planner agent:

   • Generates a plan with ordered steps (web research, internal lookups, synthesis).
   • Each step can specify a tool to use (web_search, rag, or none).

3. Orchestrator:

   • Executes each step, calling:
     • WebResearchAgent for web search notes
     • RAGAgent for internal docs notes
   • Maintains a shared state dict across steps.

4. Writer agent:

   • Consumes:
     • user query
     • web research notes
     • RAG notes (if available)
   • Generates a structured markdown report:
     • Overview
     • Key Findings
     • Implementation Notes
     • Risks & Tradeoffs
     • References

5. Git writer tool:

   • Builds a slugified filename.
   • Writes the markdown to reports/<slug>-<timestamp>.md.
   • Optionally runs git add and git commit for the new report.

6. FastAPI response:

   • Returns the plan, and the report_path where the report was saved.

---

Getting started

1. Clone and create virtual environment

git clone https://github.com/<your-username>/production-research-copilot.git
cd production-research-copilot

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate

pip install -r requirements.txt

2. Configure environment variables

Copy the example env file and set your Groq key:

cp .env.example .env

In .env:

GROQ_API_KEY=gsk_your_real_key_here

LLM_PROVIDER=groq
LLM_MODEL=llama-3.3-70b-versatile
LLM_API_BASE=https://api.groq.com/openai/v1

You can also adjust options in config.example.yaml and copy it to config.yaml if needed.

3. Prepare RAG data (optional but recommended)

Place your PDFs/markdown inside data/docs/, then run a one‑time ingestion script (not included here, but you would typically):

• Walk data/docs
• Chunk documents
• Call RAGRetriever.add_documents() with (id, chunk_text) tuples

If RAG is not configured yet, the system still works using only web (or stubbed) research.

---

Running the API server

From the project root:

uvicorn app.server:app --reload

You should see:

Uvicorn running on http://127.0.0.1:8000

Health check:

• Open http://127.0.0.1:8000/ → {"status":"ok","message":"Production Research Copilot running"}

---

Running a research request

Example using PowerShell:

$body = @{ query = "Best open-source frameworks for building production AI agents in 2026" } | ConvertTo-Json

Invoke-WebRequest `
  -Uri "http://127.0.0.1:8000/api/research" `
  -Method POST `
  -ContentType "application/json" `
  -Body $body | Select-Object -ExpandProperty Content

Or with curl:

curl -X POST "http://127.0.0.1:8000/api/research" \
  -H "Content-Type: application/json" \
  -d '{"query": "Best open-source frameworks for building production AI agents in 2026"}'

You should see a JSON response containing:

• plan: the planner’s step-by-step plan
• report_path: the filesystem path where the .md report was written

Check reports/ to see the generated markdown file.

---

Running tests

pytest tests

The tests cover:

• Planner sanity (returns a structured plan)
• Orchestrator wiring (using dummy agents to avoid external calls)

---

Configuration: swapping models and tools

LLM provider

The LLM client is implemented in app/llm_client.py and reads from Settings in app/config.py.

• Default: Groq (OpenAI‑compatible)
• To use a different provider:
  • Change LLM_PROVIDER, LLM_MODEL, LLM_API_BASE in .env
  • Ensure the endpoint is OpenAI‑compatible or extend LLMClient with a new method.

Web search tool

Located in app/tools/web_search.py.

• For local dev, you can use a stubbed implementation that returns fake results.
• To use a real search API (e.g., Tavily, Serper):
  • Add the corresponding HTTP call here.
  • Keep the rest of the system unchanged.

Vector store / RAG

Located in app/tools/rag_retriever.py.

• Default: Chroma with a pluggable embedding function.
• You can swap to:
  • Local sentence-transformers
  • Other vector DBs (Qdrant, Pinecone, etc.) by replacing this file.

---

Deployment

Docker

Build the Docker image:

docker build -t production-research-copilot .

Run locally:

docker run -p 8000:8000 --env-file .env production-research-copilot

The service will be available at http://127.0.0.1:8000.

Render / Fly.io / other platforms

• Use the provided Dockerfile.
• Set these environment variables in your platform dashboard:
  • GROQ_API_KEY
  • LLM_PROVIDER
  • LLM_MODEL
  • LLM_API_BASE
• Point incoming HTTP traffic to port 8000.

---

What I learned

Building this project surfaced several practical lessons relevant to real‑world AI/agent systems:

• Retrieval tradeoffs
  Chunk size, top‑k, and embedding choice have a big impact on answer quality and latency. For many production cases, a smaller top‑k with good chunking provides a better balance than “retrieve everything”.

• Agent design & planning
  A planner agent that produces a structured plan makes the system easier to reason about and extend. It also surfaces intermediate steps (like filtering and comparing frameworks) that would be implicit in a single prompt.

• Execution reliability
  Tool calls (web search, vector DB, git) are the main failure points, not the LLM. Simple timeouts, exception handling, and default fallbacks keep the research flow running even when some tools fail.

• Cost & latency optimization
  Using lighter models for planning and web summarization, and reserving heavier models for report writing, keeps latency and cost manageable while still delivering high‑quality output.

---

Possible extensions

Ideas for future iterations:

• UI – Add a minimal React / Next.js front‑end to trigger research jobs and view reports.
• Workspace integrations – Add tools to push reports to Notion, Confluence, Jira, or GitHub PR comments.
• Eval harness – Add synthetic tasks and use an eval model to score reports on coverage and faithfulness.
• Multi‑model routing – Use different models for planning vs writing (e.g., small Groq model for planner, larger for writer).
• Job-specific modes – Specialized prompts for domains like fintech, healthcare, or MLOps.

---

License

MIT (or your preferred license).