Commit 6b640d9
tokenize: fix Artifact.load -> Artifact.from_path rename misses (#5756)
## Summary
- PR #5727 renamed `Artifact.load` → `Artifact.from_path` in
`marin.execution.artifact` but left three call sites in
`lib/marin/src/marin/processing/tokenize/` calling the now-missing
method.
- Both `tokenize_attributes_step` and `build_levanter_store_step` blow
up at runtime with `AttributeError: type object 'Artifact' has no
attribute 'load'` the first time their `_fn` closure runs (i.e. inside
the StepRunner, not at module import).
- Rename the 3 call sites + 2 stale docstring references to match.
## Test plan
- [x] `uv run pytest tests/processing/tokenize/test_split_tokenize.py` —
18 passed (existing tests, no new ones).
- [x] `uv run pyrefly check` — clean.
- [x] `./infra/pre-commit.py --files ...` — clean.
- [x] Surfaced while running
`experiments/tokenize/all_sources_tokenize.py` on Iris — every step hit
the AttributeError immediately. Note: existing unit tests don't catch
this because they exercise `tokenize_attributes` /
`build_levanter_store` directly, not the `*_step` factory closures.
Co-authored-by: Rafal Wojdyla <ravwojdyla@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent f8b87e5 commit 6b640d9
2 files changed
Lines changed: 5 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| |||
281 | 281 | | |
282 | 282 | | |
283 | 283 | | |
284 | | - | |
| 284 | + | |
285 | 285 | | |
286 | | - | |
| 286 | + | |
287 | 287 | | |
288 | 288 | | |
289 | 289 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
70 | | - | |
| 70 | + | |
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
| |||
354 | 354 | | |
355 | 355 | | |
356 | 356 | | |
357 | | - | |
| 357 | + | |
358 | 358 | | |
359 | 359 | | |
360 | 360 | | |
| |||
0 commit comments