Skip to content

feat(ai-observability): route eval-report agent through the ai-gateway#65043

Draft
brandonleung wants to merge 1 commit into
brandon/llma-clustering-gatewayfrom
brandon/llma-eval-report-gateway
Draft

feat(ai-observability): route eval-report agent through the ai-gateway#65043
brandonleung wants to merge 1 commit into
brandon/llma-clustering-gatewayfrom
brandon/llma-eval-report-gateway

Conversation

@brandonleung

@brandonleung brandonleung commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Problem

The eval-report agent built its OpenAI client directly on OPENAI_API_KEY. Make it routable through the internal Go ai-gateway by env, with no behavior change until flipped.

Changes

  • The report agent calls build_openai_chat_client directly (the shared helper the cluster-labeling agents use): routes through the gateway when AI_GATEWAY_URL + AI_GATEWAY_API_KEY are set, else direct to OpenAI. The Cloud/DEBUG guard lives in the helper.

Rollout

Both vars unset (default) is today's behavior; setting them is the cutover. Two cutover-time items, neither active until then: confirm the gateway routes gpt-5.2, and drop the posthoganalytics callback (the gateway captures $ai_generation itself, so keeping it would double-capture).

Not in this PR

OpenAI summarization migration, stacked on top (#65044).

How did you test this code?

Agent-authored. The routing logic is covered by the helper's suite (test_llm_endpoint.py); the report-agent graph suite stays green. ruff clean. No manual run.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Built with Claude Code; mirrors the merged cluster-labeling routing change. Skills: /branch-review and /pr-descriptions. Addressed Greptile's review (removed the superfluous _get_llm wrapper, calling the helper directly). Requires human review.

@greptile-apps

greptile-apps Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
posthog/temporal/ai_observability/eval_reports/report_agent/graph.py:34-41
**Superfluous wrapper function**

`_get_llm` is now a single-line delegation to `build_openai_chat_client` with an identical signature and no added logic. By simplicity rules 3 (OnceAndOnlyOnce) and 4 (no superfluous parts), the wrapper should be deleted and the call site on line 196 updated to call `build_openai_chat_client(EVAL_REPORT_AGENT_MODEL, EVAL_REPORT_AGENT_TIMEOUT)` directly — eliminating one indirection and making the import the single statement of intent.

Reviews (1): Last reviewed commit: "feat(ai-observability): route eval-repor..." | Re-trigger Greptile

Comment thread posthog/temporal/ai_observability/eval_reports/report_agent/graph.py Outdated
Call build_openai_chat_client directly for the report agent's LLM client, so
it routes through the internal Go ai-gateway when AI_GATEWAY_URL +
AI_GATEWAY_API_KEY are set and goes direct to OpenAI otherwise. The Cloud/DEBUG
guardrail lives in the shared helper. Default-unset preserves current behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@brandonleung brandonleung force-pushed the brandon/llma-eval-report-gateway branch from be4d5e6 to c696081 Compare June 21, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant