Skip to content

Commit 9853713

Browse files
lfengadclaude
andauthored
fix(checkpoints): repin Super 4Step checkpoints to main (#144)
## Problem `distilled-inference-smoke` fails during checkpoint download, before inference ever starts: ``` 403 Forbidden Cannot access content at: https://cdn-lfs-us-1.hf.co/repos/01/58/.../be75606093db... <Error><Code>AccessDenied</Code><Message>Access Denied</Message></Error> subprocess.CalledProcessError: ['uvx', ..., 'hf@1.16.4', 'download', 'nvidia/Cosmos3-Super-Text2Image-4Step', '--revision', '1ba94110bc...'] ``` rank 0 fails → ranks 1-3 raise `RuntimeError: Rank 0 download failed` → `ChildFailedError`. Seen on #134 ([job](https://github.com/NVIDIA/cosmos-framework/actions/runs/30432072419/job/90521649702)), where both the T2I and I2V cases died this way — nothing to do with that PR's changes. ## Root cause The LFS blobs behind the two pinned commits are still served from the legacy `cdn-lfs-us-1.hf.co` bucket, and those objects were reclaimed after the repos migrated to Xet. The same files on each repo's `main` branch serve fine from `us.aws.cdn.hf.co`. Reproduced deterministically with plain `curl` (3/3 attempts): | repo | file | pinned rev | `main` | |---|---|---|---| | `Cosmos3-Super-Text2Image-4Step` | `text_tokenizer/tokenizer.json` | **403** | 200 | | `Cosmos3-Super-Image2Video-4Step` | `transformer/diffusion_pytorch_model-00001-of-00027.safetensors` | **403** | 206 | `tokenizer.json` has a different oid and size across the two revisions (`be7560…`/11422650 vs `aeb133…`/11422654), i.e. the file was rewritten after we pinned and the old blob is gone. `main` CI stays green only because its runner (`computeinstance-e01yhncr…`) has these files cached — the download command returns in 1.2 s without touching the CDN. #134 landed on `computeinstance-e01kqprb…` with a cold cache and hit the real fetch. **Any PR scheduled onto a cold-cache runner fails the same way.** ## Fix Repin both to `revision="main"`, matching the other five Cosmos3 checkpoints in the same registry. ## Scope check Swept every 40-hex `revision=` pin in the repo against the Hub with an `HF_TOKEN` (first 8 LFS files per repo, ranged GET). Only these two are affected: | repo @ pinned rev | result | |---|---| | `Cosmos3-Super-Text2Image-4Step` @ `1ba94110` | **403** | | `Cosmos3-Super-Image2Video-4Step` @ `f85d3335` | **403** | | `Cosmos3-Nano-Reasoner` @ `6406357c` | OK | | `Cosmos3-Super-Reasoner` @ `b9b716f3` | OK | | `Cosmos3-Experimental` @ `6ca42c5d` / `a3743aa1` | OK | | `Cosmos-Guardrail1` @ `d6d4bfa8` | OK | | `Wan2.1-T2V-14B`, `Wan2.2-TI2V-5B` | OK | | `BridgeData2-Subset-Synthetic-Captions`, `LIBERO_LeRobot_v3`, `bridge_lerobot_v3` | OK | `Cosmos3-Edge` already uses `revision="main"` in both `inference/common/checkpoints.py` and `configs/base/experiment/sft/models/edge_model_config.py`. ## Verification ``` $ pytest cosmos_framework/inference/args_test.py -k distilled_checkpoint_uses_published 2 passed $ pytest cosmos_framework/inference/args_test.py -k test_checkpoints 1 passed ``` `test_checkpoints` really downloads `config.json` / `transformer/config.json` for every registered checkpoint, so it confirms both `main` revisions resolve and fetch. ## Follow-up (not in this PR) The comment at `args_test.py:129-131` says the 4-step repos are gated, but the Hub now reports `gated: false, private: false` for both and they fetch anonymously. The `HF_TOKEN` skip branch in `test_checkpoints` looks droppable. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 9726697 commit 9853713

2 files changed

Lines changed: 4 additions & 4 deletions

File tree

cosmos_framework/inference/args.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1223,7 +1223,7 @@ def build_sample(self, *, model_config: Any) -> OmniSampleArgs:
12231223
s3_uri="s3://bucket1/cosmos3_vfm/cosmos3_ga_text2image_4step/",
12241224
hf=CheckpointDirHf(
12251225
repository="nvidia/Cosmos3-Super-Text2Image-4Step",
1226-
revision="1ba94110bc118f479bbd5e461e79d685d74b2554",
1226+
revision="main",
12271227
),
12281228
experiment_overrides=(
12291229
"model.config.resolution='768'",
@@ -1245,7 +1245,7 @@ def build_sample(self, *, model_config: Any) -> OmniSampleArgs:
12451245
s3_uri="s3://bucket1/cosmos3_vfm/cosmos3_ga_image2video_4step/",
12461246
hf=CheckpointDirHf(
12471247
repository="nvidia/Cosmos3-Super-Image2Video-4Step",
1248-
revision="f85d3335d2ad8b352462cecbd637aa980cec9688",
1248+
revision="main",
12491249
),
12501250
experiment_overrides=(
12511251
"model.config.resolution='480'",

cosmos_framework/inference/args_test.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -151,14 +151,14 @@ def test_checkpoints():
151151
(
152152
"Cosmos3-Super-Text2Image-4Step",
153153
"nvidia/Cosmos3-Super-Text2Image-4Step",
154-
"1ba94110bc118f479bbd5e461e79d685d74b2554",
154+
"main",
155155
"768",
156156
24,
157157
),
158158
(
159159
"Cosmos3-Super-Image2Video-4Step",
160160
"nvidia/Cosmos3-Super-Image2Video-4Step",
161-
"f85d3335d2ad8b352462cecbd637aa980cec9688",
161+
"main",
162162
"480",
163163
16,
164164
),

0 commit comments

Comments
 (0)