Repository navigation
start-all: exit when any service dies, so launchd actually restarts the stack - #84
Merged
Merged
Conversation
2026-08-30: the confirmations daemon died on this Mini. Nothing restarted it. start-all.sh carried on, launchd saw a healthy job, and the room's approval routing and message notifications were simply gone until a human noticed and ran 'launchctl kickstart'. That daemon holds :8788 and settles every Approve/Deny tap, so its death is the most expensive one in this file and was the least visible. Cause: a bare 'wait' returns only when EVERY child has exited. One dead service is invisible to it. Now each service's pid is collected and a supervisor loop exits as soon as one of them stops, killing the siblings so launchd's KeepAlive brings the whole stack back together. 'wait -n' would be the one-liner but macOS ships bash 3.2 and does not have it; polling with kill -0 works everywhere. Verified with a mock of the same structure: killing one child made the supervisor exit non-zero within the poll interval, take its siblings with it, and leave no orphans. bash -n clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
To use Codex here, create an environment for this repo. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Today the confirmations daemon died on the Mini and nothing brought it back.
start-all.shkept running, launchd saw a healthy job, and the room lost approval routing and message notifications until a human noticed and ranlaunchctl kickstart.Cause: the script ends in a bare
wait, which returns only when every child has exited. A single dead service is invisible to it. That daemon holds :8788 and settles every Approve/Deny tap, so it is the most expensive process in the file and was the least visible when it went.Fix: collect each service's pid and supervise them in a loop; the moment one stops, kill the siblings and exit non-zero so launchd's KeepAlive restarts the stack as a unit.
wait -nwould be the one-liner, but macOS ships bash 3.2, which does not have it —kill -0polling works everywhere and costs nothing at a 5s interval.Verified, not just reasoned about: a mock with the same structure and three children. Killing one made the supervisor exit within the poll interval, take its siblings with it, and leave zero orphans.
bash -nclean.Note this is the second supervision gap found today — the antfarm bridge has a
KeepAliveplist that is not loaded, so it is unsupervised too. Different repo, filed separately.@codexmb worth a look.