[nightshift] 20260424 multi-cleanup#5153
Merged
claude-nightshift[bot] merged 3 commits intomainfrom Apr 24, 2026
Merged
Conversation
…gger, dedupe branch `load_dataset_with_backoff` stopped accepting a `logger` kwarg in be63d53 when retry_with_backoff was extracted to rigging, but the two call sites in transform_conversation kept passing `logger=logger`. The kwarg was being forwarded through `**dataset_kwargs` into `datasets.load_dataset`, where `**config_kwargs` swallows it as a (nonsense) builder config option. Also: - Replace `logging.log(logging.WARNING, ...)` / `logging.info(...)` with the module-level `logger` already defined in the file. - Collapse the if/else branch in `transform_row` whose two arms ran the identical `_normalize_tool_structures` list comprehension.
ScatterWriter was re-exported from execution.py for "external callers" but had no external callers anywhere in the tree. ScatterReader had one external caller (plan.run_stage) that is already in the same file as another ScatterReader import from shuffle. Import it from shuffle directly for consistency, drop both re-exports, and hoist the remaining ListShard / MemChunk / _write_scatter import to the top of execution.py (the late placement with noqa: E402 was unnecessary - shuffle does not import from execution, so there is no cycle).
Drop three never-used helpers and their imports: levanter.shapes.conforms, levanter.optim.util.hvp, and levanter.optim.util.tree_gaussian_like. Also drop a duplicate ``new_total = self.total + total`` assignment in RunningMean.add that was already computed a few lines above.
claude-nightshift
Bot
added
agent-generated
rjpower
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three independent scout agents ran parallel cleanup sweeps across the
subprojects; this PR bundles their resulting commits.
lib/marin/src/marin(scout commit2f2afbf6c)Fixed a latent bug in
transform/conversation/transform_conversation.py: bothcall sites of
load_dataset_with_backoffwere still passinglogger=loggerafter that kwarg was removed from the helper (when
retry_with_backoffwasextracted into
rigging). The kwarg was being silently forwarded through**dataset_kwargsintodatasets.load_dataset's**config_kwargsas anonsense builder option. Also swapped six root-level
logging.log/logging.infocalls for the module-level
loggeralready defined there, and collapsed anif/elseintransform_rowwhose two arms both ran the identical_normalize_tool_structureslist comprehension.lib/zephyr/src/zephyr(scout commit00340541e)Dropped two dead module re-exports from
execution.py:ScatterWriterhad no importers anywhere in the tree.ScatterReaderhad one internal caller (plan.run_stage) that was alreadyimporting the same symbol from
zephyr.shufflein the same function —switched that caller to import solely from
zephyr.shuffle.With the re-exports gone, the late
noqa: E402shuffle import block (a relicof an earlier cycle that no longer exists —
shuffledoes not import fromexecution) hoists cleanly into the top-of-file imports.lib/levanter/src/levanter(scout commitc418a33e6)Removed three never-used helpers and a redundant recomputation:
levanter.shapes.conforms— zero callers across the monorepo.levanter.optim.util.hvp— zero callers.levanter.optim.util.tree_gaussian_like— zero callers.levanter.utils.stat_utils.RunningMean.add— dropped a second computationof
new_totalthat was already computed a few lines earlier.No behavior change.
lib/iris/src/irisScout reported no changes (permissions prevented commit). It identified two
candidates for a future run:
ScaleGroupSpecincluster/config.pyandresource_capacity_from_specincluster/constraints.py, both unused acrossthe tree.
Test plan
./infra/pre-commit.py --all-files --fixpytest tests/transform/test_conversation.py(6 passed)pytest lib/zephyr/tests/test_execution.py test_shuffle.py test_worker_group_race.py(59 passed)pytest lib/levanter/tests/test_optimizer_linear_like.py test_optimizer_config.py test_scan_stack_optimizers.py test_namo.py test_eval.py test_grad_accum.py(54 passed)