Commit cca926b
authored
[pipeline] Skip process-group stats monitor in daemon processes (#1590)
`ProcessGroupStatsMonitor.run()` spawns a `multiprocessing` subprocess
to read `/proc` for per-process-group CPU/mem/IO stats. When the
pipeline is built inside a daemon process — e.g. a worker-pool process
running a nested pipeline (a `.to()` region worker, or the old
fused-subprocess pool worker) — `proc.start()` raises `AssertionError:
daemonic processes are not allowed to have children`, which the
background-task runner logs as an ERROR (once per worker, repeatedly
noisy in MAST logs).
A daemon genuinely cannot spawn the monitor subprocess, and a per-worker
monitor would be redundant anyway: the main-process monitor already
collects stats for the whole process group (it sums over the shared
PGID). So guard the spawn — if the current process is a daemon, log a
single WARNING and return cleanly instead of letting the assertion
surface as an error.1 parent 82f5050 commit cca926b
1 file changed
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
630 | 630 | | |
631 | 631 | | |
632 | 632 | | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
633 | 639 | | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
634 | 647 | | |
635 | 648 | | |
636 | 649 | | |
| |||
0 commit comments