Skip bundle-backed code functions + surface skip detail location#22
Merged
Merged
Conversation
A code function (function_data.type == "code") can be backed by either inline source (function_data.data.type == "inline") or a compiled bundle/sandbox artifact. The inline variant carries its full code and migrates fine, but the bundle variant only references an artifact produced by the push/eval build pipeline — that artifact isn't exposed by the API and can't be recreated in the destination, so migrating it yields a function with a dangling bundle_id that can't be invoked. FunctionMigrator now detects bundle-backed code functions and skips them, recording a MigrationResult(skipped=True, skip_reason="code_bundle_not_migratable") so they appear in the skip summary (not as failures) and logging each one's name/slug so it can be re-pushed to the destination manually. Inline code functions and all other function types are unaffected. Adds tests for the detection and the migrate_batch skip/keep behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Failures are printed inline in the console summary, but skips were only shown as counts — so users couldn't tell *what* was skipped without knowing to open migration_report.json. Rather than duplicate the (potentially large) per-item list in the console, print a pointer when anything was skipped: the count plus the report path and the JSON location (detailed_breakdown.skipped) where each skipped item is recorded with its name and skip_reason. Applies to all skips (already-migrated, unresolved-dependency, and the new code_bundle_not_migratable). Adds tests for the pointer shown/hidden cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Doug Guthrie (dpguthrie)
changed the title
Collaborator
Author
|
Added a second commit: the console summary now points to where per-item skip detail lives when anything is skipped. Failures already print inline in the summary, but skips only showed as counts — users couldn't tell what was skipped without knowing to open Applies to all skip reasons (already-migrated, unresolved-dependency, and the new code_bundle_not_migratable), not just functions. Tests included; 303 unit tests green. |
Add an explicit README callout (Resource Types) explaining that code functions backed by a compiled bundle — e.g. code-based scorers/tools/tasks pushed via `braintrust push` or an eval — cannot be migrated (the API only exposes a bundle reference, not the code, and there's no re-upload path), so they are skipped and recorded in migration_report.json with skip_reason="code_bundle_not_migratable". Users should re-push them manually. Inline code functions and all other function types migrate normally. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Print the full per-item report location regardless of whether anything was skipped — it's general drill-in info (migrated / skipped / failed detail), not specific to skips. The Skipped count is already shown in the summary table, so the pointer no longer repeats it; it just points to migration_report.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Code-based scorers (and any bundled code function) migrate "successfully" but land broken. When you push a code function, the build pipeline compiles a bundle and uploads it to object storage; the function payload only references it (
bundle_id+location+ a truncatedpreview). The migration copies that reference verbatim, so in the destination thebundle_idpoints at the source org's storage and the function can't be invoked.Can we just copy the bundle? No. The compiled bundle is an output of the push/eval build pipeline; there's no API to download a built bundle and re-upload it as a new bundle in the destination org, and the function object doesn't contain the code (only a preview). So bundle-backed code functions genuinely can't be recreated by the tool.
What this does
Skip only the un-migratable ones. Code functions come in two flavors (per the functions API):
inline(function_data.data.type == "inline") — carries its fullcode+runtime_context; the runtime executes it directly. Migrates fine; kept.bundle/sandbox— references a build artifact that isn't retrievable. Skipped.FunctionMigratornow detectsfunction_data.type == "code"withdata.type != "inline"and skips it, recorded asMigrationResult(skipped=True, skip_reason="code_bundle_not_migratable")so it shows in the skip summary (not as a failure) and logs each skipped function's name/slug so you know what to re-push manually. Inline code functions and all other types are unaffected.This applies to any bundled code function (scorers, tools, tasks) — the bundle problem is the same regardless of
function_type.Tests
_is_code_bundle_functiondetection: bundle ✅ skipped, inline ✅ kept, prompt/global/sandbox/missing-data covered.migrate_batch: a mix of [bundle, inline, prompt] → bundle skipped with the reason and never sent to the create endpoint; inline + prompt migrate normally.🤖 Generated with Claude Code