Read the process table in one place, and stop spawning ps on Linux - #171
Merged
Conversation
Every caller that needed a fact about a process ran its own `ps` and treated the output as fact. A subprocess can be slow, truncated or silent, and all three look exactly like "the process is gone". Node cannot make the syscalls the Rust tool uses, so this does what Node can. On Linux there is now no subprocess at all: `/proc` carries ppid, pgid, state and starttime, which is every fact the callers ask for. On macOS `ps` stays, but the table is read once per operation rather than once per process per poll. In the teardown loop that is the difference between 240 spawns inside a 1500 ms deadline and 60. Silence is a third answer everywhere. Every query separates the fact from "the table was read and this process is not in it" from "I could not find out", with no default and no conversion that turns the last into the middle by accident. Treating silence as death requires calling `orAbsentWhenUnknown`, which greps in one command. That makes the mistake visible rather than impossible. A listing that does not contain the process that read it was truncated, not empty. `ps` always lists itself. `recovery.processStartToken` is untouched and still comes from `ps -o lstart=`. Its exact text is a contract with the Rust tool through a shared registry, so the parser takes the tail verbatim rather than re-joining split fields, which would have rewritten `Wed Sep 3` as `Wed Sep 3`. The in-memory identity is a separate branded type so the two can never be compared. Production `ps` call sites: six to three, none in a per-process poll loop.
An unreaped descendant keeps its `/proc` row and its identity on Linux, so matching on identity alone counted it as alive. The teardown would wait out its whole TERM budget for a process that had already died, and then report it as having survived a SIGKILL. That is the kill over-claiming again, in the other direction. macOS never had this, because `ps` stops listing a process the moment it exits. The two platforms disagreeing is what exposed it. Two test fixes from a real Mac run, reported by Silber.pty: macOS has the `setsid` system call but no `setsid` executable. The real process-group test spawned the binary, so on the one platform where process groups are the whole escalation story, the test could not run at all. It now uses `detached: true`, which is the same thing without the command. The zombie test asserted the Linux mechanism rather than the conclusion. On Linux the corpse keeps a row with state Z; on macOS it is dropped from the listing at once. The test now asserts what every caller depends on, which is the same on both. A single-pid query no longer reads the whole table. `hasProcessExitedForReap` sits inside poll loops that run every 25 ms, and asking about one pid should not pay for every process on the machine.
The single-process read handed the subprocess's raw stdout to the single-line parser. `ps` ends its output with a newline and that parser's `$` does not match before one, so on macOS a live process read as "field-empty" and a zombie read as NOT EXITED. The teardown would then have waited out its whole budget for a corpse. That is the corpse defect a third time, reintroduced by a second parsing path that Linux never exercised, because on Linux this read goes to `/proc`. There is one parser now. The single-process read goes through `parsePsListing`, which splits lines first and checks that the row for the pid it asked about is actually present, so both jobs are done by the code that was already tested. The other two `ps` call sites were checked for the same seam. `server.ts` and `recovery.ts` both trim before parsing; only the path added by this branch did not. Nothing caught it because every test built its input the way the parser expected. They all agreed with each other and none of them agreed with `ps`. So there is now a test that runs the real command with the real arguments and feeds the parser exactly what the subprocess wrote, newline and all. Reverting to the old behaviour fails it. Found on a real Mac by Silber.pty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The question this answers
Nathan asked: "Should we just stop using
psor is the current code alright?"The current code is not alright, and the reason is arithmetic.
The teardown polls every 25 ms and asked the operating system about each
surviving descendant separately — a
psspawn each time. Inside the 1500 msTERM budget that is up to 60 iterations:
psspawns in a 1500 ms budgetThe teardown could not meet its own deadline for a tree of four, on an idle
machine. Reading
psmore carefully does not touch that: the cost is thespawn.
What Node can do, and what it cannot
The Rust tool can replace
pswith syscalls. Node cannot, without a nativemodule. I considered that and rejected it: a globally installed CLI that needs
a compiler at install time, or prebuilt binaries per platform × architecture ×
Node ABI, buys a smaller bug than it costs. Existing npm packages mostly spawn
psthemselves, which moves the defect behind a dependency.So this does the two things Node can do:
On Linux there is now no subprocess at all.
/proc/<pid>/statcarries ppid,pgid, state and starttime — every fact the callers ask for.
On macOS
psstays, read once per operation rather than once per processper poll. In the teardown loop that is 240 spawns down to 60.
Production
pscall sites: 6 → 3, and none of the three is inside aper-process poll loop.
proc-table.tsserver.tsrss/pcpufor one session's stats, off Linux onlyrecovery.tsThe defect that only a cross-platform run could find
An unreaped descendant kept its
/procrow and its identity on Linux, so theteardown counted a corpse as a survivor. It waited out the whole TERM budget
for a process that could not respond, then reported it as having survived a
SIGKILL. That is a dishonest kill in the other direction — the thing the
kill PR exists to remove, pointing the opposite way.
It was pre-existing, not introduced by this work. The old code matched on a
start token read from
/proc, which a zombie still has.macOS never had it, because libproc drops the corpse the moment it exits —
the same libproc behaviour that is the bug being fixed above. So:
Silber.ptyran both branches against the same zombie on the same Mac at thesame moment, got a
Zrow from one andNotPresentfrom the other, andreported the disagreement rather than either result. Chasing why is what found
this.
A control confirms it is a finding rather than a claim: revert the fix and the
test fails with
an exited but unreaped child still reads as a live descendant.What a real Mac has already proved
Silber.ptyran the gate on the only Mac holding both branches:zero
pscalls;darwin:Thu Sep 3 13:35:16 2026, double space intact;ppid,pgid, status and start time from libproc matched/bin/psexactly.
Outstanding on that gate: libproc refuses an unreaped child
(
PROC_PIDTBSDINFOreturns 0 of 136 bytes withESRCH) whileproc_listpidsstill lists it.
KERN_PROC_PIDanswers. That fix is in progress and the gatewill be rerun against it.
What this costs on macOS, stated so nobody finds it later
Silber.ptymeasured the socket-owner kill proof on a Mac: two kills plus bothstarts in 4.16 seconds, making 120
pscalls — 115 whole-table reads and 5lstarttoken reads.That is option d working as designed, and a large improvement on one spawn per
process per poll. It is still 115 against the Rust tool's zero. Closing that
gap needs a native module, which for a globally installed CLI costs more than
the bug. The number is here so it is a known trade rather than a discovery.
Silence is a third answer
Every query returns an
Answer, separating the fact fromnot-present— thetable was read and this process was not in it — from
unknown. No default,no direct unwrap, no conversion that loses the distinction. A caller that
wants silence to mean death calls
orAbsentWhenUnknown, which is long onpurpose and greps in one command.
This does not make the mistake impossible, and I would rather say so than
imply otherwise. It makes it visible in review.
A listing that does not contain the process that read it was truncated, not
empty.
psalways lists at least itself. That one comparison turns a silentor half-written listing into
unknowninstead of "nothing exists". A controlrun confirms it bites: remove the check and two tests fail.
The one
psthat must stay, and whyrecovery.processStartTokenis written into session metadata and read back bythe Rust tool from the same registry. Its exact text — including the two
spaces
ps -o lstart=puts before a single-digit day — is a contract betweentwo programs, not an implementation detail.
My first parser re-joined whitespace-split fields and would have quietly
rewritten
Wed Sep 3asWed Sep 3. Nothing would have failed until a Macupgraded and stopped recognising its own sessions. The parser now takes the
tail verbatim, and the in-memory identity is a separate branded type so the
two cannot be compared by accident.
That call is one per session lookup, never in a loop, and its failure already
means "cannot confirm" rather than "gone". It is documented in place.
What is tested, and what is not
1655 passed, 2 failed. Both failures are in
tests/shells.test.ts, one forfishand one forzsh; neither shell is installed on the machine I ran on,and running that file on unmodified
maingives the same two failures and noothers.
12 new tests pin the three answers, the truncation guard, the verbatim
lstarttext, the
/procparser against a comm containing spaces and brackets, and —against the real machine — this process, a pid that cannot exist, and a real
zombie, which must read as present but not running.
Not tested here: the macOS path. Every test above ran on Linux, where the
psreader is not the one in use. Thepsparser itself is covered by thelisting tests, which are platform-independent, but no macOS machine has run
this code and it should be exercised on one before merging.