You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Split from #4 and being implemented in #27 by @M4h1m4. Follows the shape of #12.
A CitationCorrectnessScorer in rai_toolkit/scorers/ that checks whether citations in a response resolve to retrieved context that actually supports the cited claim. This catches two failures GroundednessScorer does not: a citation pointing at a source absent from the context, and a supported claim attributed to the wrong source. A response can be fully grounded and still cite incorrectly.
Cited spans are emitted verbatim and verified against the row before being trusted, reusing _verified_evidence_spans as-is (post-fix: accept normalized groundedness evidence spans #18 it handles whitespace/quote normalization with verbatim mapping back to row offsets).
Keep the existing {response_span, context_span} span shape and carry the citation marker alongside it.
Rows with no retrieved context return assessed=False, not a defaulted zero.
A response with no citations returns un-assessed with skipped: "no_citations". Citation presence enforcement is a separate policy metric, out of scope here.
Tests mock _call_judge, no key or network needed.
Retrieval relevance and context precision/recall stay on #4.
Split from #4 and being implemented in #27 by @M4h1m4. Follows the shape of #12.
A CitationCorrectnessScorer in rai_toolkit/scorers/ that checks whether citations in a response resolve to retrieved context that actually supports the cited claim. This catches two failures GroundednessScorer does not: a citation pointing at a source absent from the context, and a supported claim attributed to the wrong source. A response can be fully grounded and still cite incorrectly.
Conventions agreed on the #4 thread:
Retrieval relevance and context precision/recall stay on #4.