Skip to content

feat: add run.sh to f1_radio_rag - #19

Merged
LeoRoccoBreedt merged 2 commits into
mainfrom
lrb/example-run-scripts
Jul 1, 2026
Merged

feat: add run.sh to f1_radio_rag#19
LeoRoccoBreedt merged 2 commits into
mainfrom
lrb/example-run-scripts

Conversation

@LeoRoccoBreedt

Copy link
Copy Markdown
Collaborator

Summary

Adds CI support to use-cases/f1_radio_rag — the first example to go through the new PR test workflow (from PR #16).

What changed

  • run.sh — runs ingest then ask, logging one RAG trace to Opik. Lightweight alternative to the full eval suite for CI.
  • config.py — reads OPIK_EXAMPLES_MODEL so CI routes to a cheap model (openai/gpt-4o-mini). Removes LLM_READY guard — missing keys now fail loudly via litellm rather than silently skipping.
  • cli.py — removes the LLM_READY dry-run branch in ask.

Tested locally

Ran bash run.sh with real credentials. Trace appeared in Opik workspace under project f1-radio-rag.

Depends on PR #16 merging first for the PR test workflow to fire.

🤖 Generated with Claude Code

@LeoRoccoBreedt
LeoRoccoBreedt changed the base branch from main to lrb/ci-workflows June 26, 2026 10:45
@LeoRoccoBreedt
LeoRoccoBreedt changed the base branch from lrb/ci-workflows to main June 26, 2026 10:50
@LeoRoccoBreedt
LeoRoccoBreedt requested a review from fschlz June 26, 2026 11:54
@LeoRoccoBreedt
LeoRoccoBreedt marked this pull request as ready for review June 26, 2026 11:54
@LeoRoccoBreedt

Copy link
Copy Markdown
Collaborator Author

@fschlz

fschlz commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Review against current main (PR #16 merged as f48e7d8, so the CI design this branch was cut from has since changed). This needs a rebase + small rework before it can go green.

1. Drop the .github/** and CONTRIBUTING.md changes — they're already on main

This branch predates the #16 merge and re-adds the CI under the old names:

This PR adds Already on main as
.github/workflows/example-compliance-check.yml compliance.yml
.github/workflows/pr-examples-test.yml pr-test.yml
.github/workflows/scheduled-examples.yml scheduled.yml
.github/ci-examples.json scheduled.json
CONTRIBUTING.md (run_examples.sh / "CI always has credentials") reconciled version

Merging as-is would create duplicate workflows running side-by-side and revert the CONTRIBUTING reconciliation. After git rebase origin/main, the diff should be only the three f1 files.

2. Keep these — they're good

  • use-cases/f1_radio_rag/run.sh (already the correct name)
  • config.py reading OPIK_EXAMPLES_MODEL — fixes the exact gap where f1 hardcoded the full-price model 👍

3. Restore a dry-run path — otherwise the merged pr-test.yml fails this PR

pr-test.yml now has a secrets-free dry-run job: it runs bash run.sh with no API keys and expects exit 0 (the only signal fork PRs get). This PR removes the LLM_READY guard so askrag.answer → litellm runs unconditionally → with no key it raises → set -e → job fails.

Fix: in ask, when no LLM key is set, print the retrieved context + a [DRY RUN] would summarise with <model> line and return (the use-case template does exactly this). ingest should be fine creds-free (local Chroma) — worth confirming.

Verify after rework (no credentials)

cd use-cases/f1_radio_rag && bash run.sh                 # must exit 0 in DRY_RUN
uv run ruff check . && uv run ruff format --check .       # the lint job checks both

With creds set, bash run.sh should log a trace (the live-run job).

@fschlz fschlz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LeoRoccoBreedt and others added 2 commits July 1, 2026 13:57
- run.sh: CI entrypoint — exports OPIK_PROJECT_NAME, uv sync, ingest + ask
- config.py: GEN/JUDGE/OPTIMIZER read OPIK_EXAMPLES_MODEL (CI cheap model),
  default anthropic/claude-sonnet-4-6
- cli.py: gate `ask` on DRY_RUN (the CONTRIBUTING.md dry-run convention) so the
  secrets-free CI job exits 0; drop the undocumented ANTHROPIC-only LLM_READY gate
- README: document run.sh, OPIK_EXAMPLES_MODEL, and dry-run behaviour

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop the undocumented ANTHROPIC-only LLM_READY gate and key the `run` command
off DRY_RUN, matching the dry-run convention documented in CONTRIBUTING.md, so
newly scaffolded examples stay provider-agnostic (any OPIK_EXAMPLES_MODEL).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@LeoRoccoBreedt
LeoRoccoBreedt force-pushed the lrb/example-run-scripts branch from b418144 to 2d1530d Compare July 1, 2026 11:58
@LeoRoccoBreedt
LeoRoccoBreedt merged commit 815e862 into main Jul 1, 2026
8 checks passed
@LeoRoccoBreedt
LeoRoccoBreedt deleted the lrb/example-run-scripts branch July 1, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants