feat: add run.sh to f1_radio_rag - #19
Conversation
|
Review against current 1. Drop the
|
| This PR adds | Already on main as |
|---|---|
.github/workflows/example-compliance-check.yml |
compliance.yml |
.github/workflows/pr-examples-test.yml |
pr-test.yml |
.github/workflows/scheduled-examples.yml |
scheduled.yml |
.github/ci-examples.json |
scheduled.json |
CONTRIBUTING.md (run_examples.sh / "CI always has credentials") |
reconciled version |
Merging as-is would create duplicate workflows running side-by-side and revert the CONTRIBUTING reconciliation. After git rebase origin/main, the diff should be only the three f1 files.
2. Keep these — they're good
use-cases/f1_radio_rag/run.sh(already the correct name)config.pyreadingOPIK_EXAMPLES_MODEL— fixes the exact gap where f1 hardcoded the full-price model 👍
3. Restore a dry-run path — otherwise the merged pr-test.yml fails this PR
pr-test.yml now has a secrets-free dry-run job: it runs bash run.sh with no API keys and expects exit 0 (the only signal fork PRs get). This PR removes the LLM_READY guard so ask → rag.answer → litellm runs unconditionally → with no key it raises → set -e → job fails.
Fix: in ask, when no LLM key is set, print the retrieved context + a [DRY RUN] would summarise with <model> line and return (the use-case template does exactly this). ingest should be fine creds-free (local Chroma) — worth confirming.
Verify after rework (no credentials)
cd use-cases/f1_radio_rag && bash run.sh # must exit 0 in DRY_RUN
uv run ruff check . && uv run ruff format --check . # the lint job checks bothWith creds set, bash run.sh should log a trace (the live-run job).
- run.sh: CI entrypoint — exports OPIK_PROJECT_NAME, uv sync, ingest + ask - config.py: GEN/JUDGE/OPTIMIZER read OPIK_EXAMPLES_MODEL (CI cheap model), default anthropic/claude-sonnet-4-6 - cli.py: gate `ask` on DRY_RUN (the CONTRIBUTING.md dry-run convention) so the secrets-free CI job exits 0; drop the undocumented ANTHROPIC-only LLM_READY gate - README: document run.sh, OPIK_EXAMPLES_MODEL, and dry-run behaviour Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop the undocumented ANTHROPIC-only LLM_READY gate and key the `run` command off DRY_RUN, matching the dry-run convention documented in CONTRIBUTING.md, so newly scaffolded examples stay provider-agnostic (any OPIK_EXAMPLES_MODEL). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
b418144 to
2d1530d
Compare
Summary
Adds CI support to
use-cases/f1_radio_rag— the first example to go through the new PR test workflow (from PR #16).What changed
run.sh— runsingestthenask, logging one RAG trace to Opik. Lightweight alternative to the full eval suite for CI.config.py— readsOPIK_EXAMPLES_MODELso CI routes to a cheap model (openai/gpt-4o-mini). RemovesLLM_READYguard — missing keys now fail loudly via litellm rather than silently skipping.cli.py— removes theLLM_READYdry-run branch inask.Tested locally
Ran
bash run.shwith real credentials. Trace appeared in Opik workspace under projectf1-radio-rag.Depends on PR #16 merging first for the PR test workflow to fire.
🤖 Generated with Claude Code