Skip to content

ooo qa: MCP tool path defects — unpersisted qa sessions, host argument coercion, verdict/score disagreement, unguarded inputs #2310

Description

@Q00

Context

A Codex-hosted ooo qa run reported "the Ouroboros QA tool call itself failed" (a retry succeeded), then ouroboros_session_status for the returned qa-f7e392ab id answered Session not found. Reproduced on main (d5c2f9f) with a raw JSON-RPC stdio probe against ouroboros mcp serve --runtime claude-cli --llm-backend claude_code:

  • Server boots and registers 36 tools including ouroboros_qa in ~6 s — "MCP does not come up" is not reproduced.
  • ouroboros_session_status(session_id="qa-…")Session not found: No events found for session — always, because QAHandler.handle generates qa_session_id but never persists anything (src/ouroboros/mcp/tools/qa.py).
  • ouroboros_qa with pass_threshold: "0.8" or iteration_history: "[]" (strings, a common serialization from GPT-family hosts) is rejected by inputSchema validation before the handler runs — the whole tool call fails.
  • A valid call takes ~18 s for a trivial artifact through the claude CLI backend; adversarial QA on a real app can exceed the 60 s default host tool timeout on Codex.

Handler-level defects reproduced with a faked LLM adapter:

  • LLM verdict string overrides the score: {"score":0.42,"verdict":"pass"} renders Score: 0.42 / 1.00 [PASS], Verdict: pass, while meta says passed: false, loop_action: continue — three surfaces disagree; qa-judge.md says score decides.
  • Non-dict iteration_history entries raise AttributeError out of handle() (no Result.err), since build_qa_subagent runs above the try:.

This issue is the umbrella for a series of single-subsystem PRs; each PR references it and fixes exactly one defect.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugReproducible defect or broken behaviorneeds-designMulti-PR epic or architectural change, needs human planning

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions