Use AI Gateway LLM providers for evaluation monitors

[nadheesh]

Overview

Currently, evaluation monitors allow users to configure custom LLM API keys directly, storing encrypted credentials in the database. This needs to be replaced with the org-level AI Gateway LLM provider pattern already used by agents.

Goals

• Remove custom LLM credential input from evaluation monitors
• Allow monitors to use LLM providers registered at the org level via the AI Gateway (same as agents)
• Store proxy API keys in OpenBao key manager (not encrypted in DB)
• Create a monitor-level LLM proxy to enable per-monitor guardrails (rate limiting, etc.)
• Support all LLM providers supported by the gateway (OpenAI, Anthropic, Gemini, Mistral, AWS Bedrock, Azure OpenAI, Azure AI Foundry)
• Disable LLM-judge evaluators in the frontend until an LLM provider is configured for the monitor

Approach

Mirror the existing agent LLM provider pattern:

1. Org-level LLM providers → per-monitor LLM proxies deployed to the gateway
2. Proxy API keys stored in OpenBao, resolved at workflow execution time
3. Injected as LiteLLM-compatible env vars into the evaluation job container
4. New monitor_llm_mapping junction table to track monitor ↔ proxy associations

Impact

• Backend: New DB migration, repository, and proxy lifecycle management in monitor manager service
• Executor: Read proxy secrets from OpenBao at runtime, pass as workflow parameters
• Frontend: Replace credential input form with an LLM provider picker; disable LLM-judge when no provider is set

[isamauny]

I would amend this to open it to support any AI Gateway I may already have for monitors. So yes, support the LLM providers we have in the list (that I may already have defined) but that implies deployment in our GW only.
We should support 3rd party GWs (including the Classic GW) by simply allowing users to specify a URL and APIkey/OAUth credentials.