Skip to content

Commit c993608

Browse files
provide full version of health check (communication testing), and also fix the manually kill detection logic
1 parent 75570eb commit c993608

3 files changed

Lines changed: 546 additions & 104 deletions

File tree

flagscale/runner/elastic/diagnostic.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@
2828
"process killed": "ProcessKilled: Training process was killed.",
2929
"killed by signal": "ProcessKilled: Process was killed by signal.",
3030
"terminated by signal": "ProcessKilled: Process was terminated by signal.",
31+
"keyboardinterrupt": "ProcessKilled: MANUAL KILL - Keyboard interrupt detected",
32+
"sigint": "ProcessKilled: MANUAL KILL - SIGINT signal received",
33+
"sigterm": "ProcessKilled: MANUAL KILL - SIGTERM signal received",
3134
"segmentation fault": "SegmentationFault: Process crashed due to memory access error.",
3235
"core dumped": "CoreDump: Process crashed and dumped core.",
3336
# CUDA errors

0 commit comments

Comments
 (0)