Skip to content

feat: CI support for tracing_finetuned_models - #21

Merged
LeoRoccoBreedt merged 3 commits into
mainfrom
lrb/tracing-finetuned-models-ci
Jul 1, 2026
Merged

feat: CI support for tracing_finetuned_models#21
LeoRoccoBreedt merged 3 commits into
mainfrom
lrb/tracing-finetuned-models-ci

Conversation

@LeoRoccoBreedt

@LeoRoccoBreedt LeoRoccoBreedt commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds run.sh — loads distilgpt2 from HuggingFace by default so CI runs without CometML registry credentials; sends real traces to Opik when OPIK_API_KEY + OPIK_WORKSPACE are set
  • Introduces USE_MODEL_REGISTRY env var to explicitly opt in to downloading from the CometML Model Registry (requires COMET_API_KEY, COMET_WORKSPACE)
  • Updates use_registered_model.py and tracing_finetuned_models.ipynb to use the new flag — registry metadata only attached to traces when USE_MODEL_REGISTRY=true

Test plan

  • Ran locally without env vars — HuggingFace fallback, no crash
  • Ran locally with OPIK_API_KEY + OPIK_WORKSPACE — traces visible in Opik
  • CI passed against lrb/ci-workflows

🤖 Generated with Claude Code

@LeoRoccoBreedt
LeoRoccoBreedt changed the base branch from lrb/ci-workflows to main June 29, 2026 13:58
@fschlz

fschlz commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Review against current main (PR #16 merged) — looks good.

run.sh is correctly named (set -e + export OPIK_PROJECT_NAME ✓), and the USE_MODEL_REGISTRY flag with a HuggingFace distilgpt2 fallback means it runs creds-free — exactly what the merged secrets-free dry-run job needs.

Confirms before merge:

  1. git rebase origin/main.
  2. The dry-run job runs bash run.sh creds-free → loads distilgpt2 + CPU inference. Confirm it finishes under the job's timeout-minutes: 15 (torch + transformers install + model download on a cold cache is the risk).
  3. uv run ruff check . && uv run ruff format --check . clean (the lint job checks format too).
  4. The notebook re-save normalized some unicode escapes (arrows etc.) — harmless, just noting it in case it inflates the diff.

Make the guide run end-to-end in notebook CI (test-notebooks.yml) without a GPU
or training:

- Gate Section 1 (fine-tune + register) behind RUN_TRAINING (default off): CI and
  default runs skip training; set RUN_TRAINING=true locally to train on a GPU.
- Section 2 loads base distilgpt2 from HuggingFace (USE_MODEL_REGISTRY off) and
  Section 3 traces inference to Opik — the live path CI exercises.
- Rework the credentials cell: read OPIK_/COMET_ keys from the environment, only
  prompt in an interactive session (never in CI), and require Comet keys only
  when training or downloading from the registry.
- Drop run.sh: notebook examples are run.sh-exempt and executed by
  test-notebooks.yml; a run.sh made the folder match both workflows (double run).

Verified: `uv run ipython tracing_finetuned_models.ipynb` runs on CPU and exits 0
(Section 1 skipped, Section 2/3 trace).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@LeoRoccoBreedt
LeoRoccoBreedt force-pushed the lrb/tracing-finetuned-models-ci branch from 1675a10 to aee40ec Compare July 1, 2026 15:15
@LeoRoccoBreedt
LeoRoccoBreedt marked this pull request as ready for review July 1, 2026 16:32
@LeoRoccoBreedt
LeoRoccoBreedt merged commit d31accb into main Jul 1, 2026
7 checks passed
@LeoRoccoBreedt
LeoRoccoBreedt deleted the lrb/tracing-finetuned-models-ci branch July 16, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants