Skip to content

Refactor and add timers in training#632

Merged
zhuzilin merged 4 commits intoTHUDM:mainfrom
fzyzcjy:feat/add_timer
Oct 30, 2025
Merged

Refactor and add timers in training#632
zhuzilin merged 4 commits intoTHUDM:mainfrom
fzyzcjy:feat/add_timer

Conversation

@fzyzcjy
Copy link
Collaborator

@fzyzcjy fzyzcjy commented Oct 29, 2025

No description provided.

self.add(name, elapsed_time)
del self.start_time[name]
if torch.distributed.get_rank() == 0:
print(f"Timer {name} end (elapsed: {elapsed_time:.1f}s)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you remove those 2 prints? I'm afraid this will cause too much print in the log.

Copy link
Collaborator Author

@fzyzcjy fzyzcjy Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally find it pretty useful since it "splits" the log into sections, and this only introduce ~10 lines of log while one step may have ~100 (or ~1000) lines of log from sglang rollout, but if you like I can remove it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, make sense.

@zhuzilin zhuzilin merged commit b6ce4f3 into THUDM:main Oct 30, 2025
4 checks passed
llltttwww pushed a commit to llltttwww/slime that referenced this pull request Nov 30, 2025
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>
Yangruipis pushed a commit to rednote-ai/slime that referenced this pull request Feb 28, 2026
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants