[Feature] Move hallucination detection into a response-stage signal

[subin9]

Related #3272

#3335 has landed, so this is one commit rebased onto main.

Purpose

The hallucination half of #3272, on the machinery #3335 introduced.
routing.signals.hallucination declares the observation, it is scored once per response from
the selected recipe's rules before any plugin runs, and the hallucination plugin of the
selected decision only enforces on it. An answer that could not be checked is unavailable
rather than clean: hallucination_context_unavailable without grounding context,
hallucination_evaluation_failed when the detector failed or was never provisioned.

Two things a reviewer should weigh. Declaring a rule provisions the detector even when no
decision enables the plugin, so an eligible response pays one inference wherever a rule is
declared. A configuration with no rule keeps the plugin-owned path, reported at load.

Test Plan

• make e2e-test E2E_PROFILE=hallucination
• unit and lint through make agent-ci-lint AGENT_BASE_REF=origin/main
• make test-semantic-router

Test Result

Local gates pass on the rebased head, and the router unit suite reports no failures.
TestHallucinationSignalReadsTheSelectedRecipeRules fails when the rules come from the root
config instead of the selected recipe, and TestHallucinationSignalDrivesTheSelectedDecisionPlugin
asserts the detector runs exactly once against a real endpoint server. The hallucination E2E
profile was not run locally, so its lane on this head is the first evidence for it.

Not covered here

Streaming responses are still not scored, for this signal or for the response jailbreak signal
in #3335.

The hallucination E2E profile moves from selection: manual to selection: pr. Its first run
OOMKilled the router during fact-check initialization under the 2Gi limit the profile carried
while it was manual, so it now asks for 8Gi, next to the 6Gi jailbreak-onerror needs for the
embedding load alone. The detector under test is remote, but the router still loads the startup
embedding model and the mmBERT-32K fact-check classifier.

The detector reads an answer only as far as its 512-token window reaches, and a longer answer is
reported as clean rather than unverified. That is the behaviour on main, where the plugin owns
detection, and this PR moves the call without changing it. Measured on main with the real model:
with the false sentence at the end of the answer, detection holds at 433 words and is lost at 553. It is a per-surface fix in the shape of #3206, #3334 and #3369, so it belongs in its
own issue rather than here.

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	a340a16
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/6a9ee860de2f66000896921d
😎 Deploy Preview	https://deploy-preview-3531--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 34.17%. Comparing base (aff932e) to head (c95a535).
⚠️ Report is 233 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3531      +/-   ##
==========================================
+ Coverage   33.93%   34.17%   +0.24%     
==========================================
  Files          20       20              
  Lines        2959     2888      -71     
==========================================
- Hits         1004      987      -17     
+ Misses       1849     1795      -54     
  Partials      106      106              

Flag	Coverage Δ
operator	34.17% <ø> (+0.24%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Xunzhuo]

Reviewed the hallucination-only commit on top of #3335. The response observation is recipe-scoped, runs once before the selected plugin enforces, and keeps missing context or detector failure distinct from a clean result; the focused unit and E2E coverage matches that lifecycle. I do not see a new blocker in this delta. This remains gated on #3335 landing, a rebase to current main, and the remaining exact-head hallucination and operator lanes completing.

[FAUST-BENCHOU]

pls resolve conflicts

[subin9]

Working on my final check matching with jailbreak :)

[Xunzhuo]

Could we send this request through MoM or auto here? openai/gpt-oss-20b is a concrete backend model, so it intentionally bypasses recipe-local signals, decisions, and plugins. The exact-head hallucination lane demonstrates the result: all five cases return empty decision, warning, and matched-hallucination headers. As written, the newly required E2E cannot exercise the signal/plugin path. Please switch it to a routing entrypoint and get this lane green.