Skip to content

Commit 6ad8c3d

Browse files
Fix: Dataloader restarting on second resume (#3908)
PR fixing dataloader restarting on second resume: #3907. ## Fix datasets < 4.8.0. ## Note It should be enough to do datasets < 5.0.0. I have been a little more conservative to be consistent with https://github.com/pytorch/torchtitan/blob/51c197c86d7c703da96f666d5a7dbd5432b4afbf/.ci/docker/requirements.txt#L2
1 parent 51c197c commit 6ad8c3d

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ dependencies = [
1414
"torchdata>=0.8.0",
1515

1616
# Hugging Face integrations
17-
"datasets>=3.6.0",
17+
"datasets>=3.6.0,<4.8.0",
1818
"tokenizers>=0.15.0",
1919
"safetensors",
2020

0 commit comments

Comments
 (0)