feat(score_traces): offline scoring of production traces with the Opik SDK

[LeoRoccoBreedt]

What

A new opik-examples entry — scripts/score_traces/ — that scores existing production
traces in an Opik project offline and logs the feedback scores (with reasons) back onto
those traces, using the same judge/metric classes and variable mappings an online
evaluation rule would use. Designed so teams whose GenAI gateway isn't yet reachable from
the Opik platform can run evals on a cron now and migrate to online rules later with no
re-authoring (same score names, same variable mapping).

How it works

Composes three public SDK primitives in a loop:
search_traces → metric.score() → log_traces_feedback_scores (one batched write).

• model.py — judge-model / gateway wiring (LiteLLM; importable without a gateway)
• metrics.py — the EVALS list: hallucination (preset), relevance (G-Eval), exact_match (custom BaseMetric)
• paths.py — resolves a dotted variables mapping into each metric's score() kwargs
• score_traces.py — the runner (windowed search → score → batched log); per-eval error isolation; DRY_RUN-safe
• utils/seed_traces.py — seeds ~10 hardcoded Q&A traces (no LLM) for an end-to-end demo
• tests/test_score_traces.py — 18 unit tests (LLM mocked)

Notes

• Runs as plain scripts (uv run python score_traces.py / utils/seed_traces.py) — no wheel/entry-point packaging.
• DRY_RUN-safe: with no OPIK_API_KEY/OPIK_WORKSPACE, both entry points exit 0 without touching the network (the CI contract).
• G-Eval is fed a labeled INPUT:/OUTPUT: payload so relevance is judged against the question (G-Eval's score() takes a single string).
• Docs cover judge auth (the LiteLLM openai/ route also needs OPENAI_API_KEY), never committing OPIK_API_KEY (prefer a service account), and pipeline/CI env config.

Testing

uv run pytest → 18 passed; uv run ruff check . clean; both entry points verified DRY_RUN-safe (exit 0, no network).

🤖 Generated with Claude Code