feat(sdk): W&B offline export migration — ExportDirSource + from_wandb_export.py (GLA-834 / GLA-610)#113
Merged
Merged
Conversation
…b_export.py (GLA-825) - `sdk/openrunner/migrate.py`: refactor into WandbRunSource protocol with LiveAPISource (preserves existing behaviour) and ExportDirSource (new). ExportDirSource walks .tar.gz / .tgz / directory, parses config.yaml/json, metrics.jsonl/.parquet, summary.json, artifacts/, media/. Adds `migrate_from_wandb_export()` public entry-point. Shared `_import_records` pipeline handles idempotent re-import (update instead of duplicate on same display_name). - `sdk/openrunner/cli.py`: adds `--from-wandb-export <path>` flag to `openrunner migrate` command. - `examples/migrate-from-wandb/from_wandb_export.py`: standalone CLI wrapper for the offline import path — thin argparse shim over migrate_from_wandb_export. - `examples/migrate-from-wandb/README.md`: usage guide, prerequisites, export layout, and what-gets-imported table. - `sdk/tests/test_migrate.py`: unit tests for ExportDirSource and migrate_from_wandb_export (no network, no wandb pkg required). - `sdk/tests/fixtures/wandb-export-mini/`: two-run fixture directory used by the new tests. - `docs/migrate-from-wandb.md`: clarify Reports/Sweeps limits and update feature-comparison table. Co-Authored-By: Paperclip <noreply@paperclip.ing>
tarfile.extractall filter='data' requires Python 3.12+; detect extraction_filter attribute and skip the kwarg on older runtimes. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ExportDirSourcetosdk/openrunner/migrate.py— reads W&B export tarballs/dirs without thewandbpackage.migrate_from_wandb_export()top-level function + CLI verbopenrunner migrate-from-wandb-export.examples/migrate-from-wandb/from_wandb_export.pystandalone script for zero-install usage.tarfile.extractallcompat on Python 3.10/3.11 (filter=kwarg is 3.12+).LiveAPISourceparity preserved.Test plan
sdk/tests/test_migrate.py— 20 tests, all green (including 2-run tarball round-trip + idempotent reimport)python3 examples/migrate-from-wandb/from_wandb_export.py --helpruns cleanf5ec7a9c) to handle non-pusher gate review per GLA-680Closes #GLA-834. Re-delivers GLA-610 acceptance criteria.
🤖 Generated with Claude Code