Commit 52f3986
authored
fix num_tokens in JsonlDataset (InternLM#1567)
1 parent 0404287 commit 52f3986
1 file changed
Lines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
400 | 400 | | |
401 | 401 | | |
402 | 402 | | |
| 403 | + | |
| 404 | + | |
403 | 405 | | |
404 | 406 | | |
405 | 407 | | |
| |||
0 commit comments