Skip to content

Commit bb23dd2

Browse files
authored
fix(kernel-cli): log fatal exits from daemon-entry before terminating (#966)
## Summary - Add process-level handlers in `daemon-entry.ts` for `uncaughtException`, `unhandledRejection`, `SIGHUP`, and `exit`. - Each handler synchronously appends a line to `daemon.log` before the process exits. - Handlers install at module load, before `main()` runs, so early kernel-init failures also leave a fingerprint. ## Why `daemon-entry` runs with `stdio: 'ignore'` under the CLI spawner (see `daemon-spawn.ts`). Node's default behaviour on `uncaughtException` / `unhandledRejection` (print stack to stderr, exit 1) therefore writes to nowhere, and the operator sees only that the daemon vanished — no trace in `~/.ocap*/daemon.log`. I've hit two silent daemon deaths in the last few weeks (matcher daemon disappearing mid-rehearsal, services daemon disappearing between registration and the next call), and in both cases the log ended cleanly on the last successful message with no shutdown line, no error, no stack trace. Debugging cost real time each time. With this change, every terminating path now leaves at least one line: ``` [timestamp] [error] Uncaught exception (about to exit): <stack> [timestamp] [error] Process exiting (code=1). ``` or ``` [timestamp] [error] SIGHUP received; exiting. [timestamp] [error] Process exiting (code=0). ``` Log-write itself is wrapped in try/catch so a broken log file doesn't mask the original exit cause. ## Test plan - [x] `yarn workspace @metamask/kernel-cli test:dev:quiet --run` — all package tests pass. - [x] `yarn workspace @metamask/kernel-cli lint` — clean. - [ ] CI green. - [ ] Manual (in a downstream branch): induce an uncaught exception in a scratch daemon under a temp \$OCAP_HOME, confirm the log line lands and the process exits 1. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Observability-only changes to daemon exit paths; SIGHUP now logs and exits 0 explicitly instead of the default silent terminate, with no auth or data-handling impact. > > **Overview** > **Daemon fatal-path visibility** when the CLI spawns `daemon-entry` with `stdio: 'ignore'`: uncaught errors and signals no longer vanish with no record in `daemon.log`. > > `daemon-entry` now builds the file logger at **module load** (before `main()`), registers `installFatalHandlers()` immediately, and logs then exits on **`uncaughtException`**, **`unhandledRejection`**, and **`SIGHUP`**, with an **`exit`** handler that always writes a final line. The log transport switches from dynamic `require('node:fs')` to **`appendFileSync`** so fatal handlers flush synchronously. CHANGELOG documents the fix under Unreleased. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 70b96eb. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent f309811 commit bb23dd2

2 files changed

Lines changed: 77 additions & 12 deletions

File tree

packages/kernel-cli/CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2020
### Fixed
2121

2222
- `kernel daemon start` refuses to start when another daemon is already listening on the same Unix socket, instead of unlinking the socket and orphaning the running process ([#952](https://github.com/MetaMask/ocap-kernel/pull/952))
23+
- Daemon fatal-path visibility: `daemon-entry` now installs handlers for `uncaughtException`, `unhandledRejection`, `SIGHUP`, and `exit` that append a synchronous fingerprint line to `daemon.log` before terminating ([#966](https://github.com/MetaMask/ocap-kernel/pull/966))
24+
- Without these, silent daemon deaths under `stdio: 'ignore'` (the CLI's default spawn mode) left no trace in the log; the operator saw only that the daemon was gone. Every terminating path now leaves at least one line.
2325

2426
## [0.1.0]
2527

packages/kernel-cli/src/commands/daemon-entry.ts

Lines changed: 75 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,31 @@ import { startDaemon } from '@metamask/kernel-node-runtime/daemon';
44
import type { DaemonHandle } from '@metamask/kernel-node-runtime/daemon';
55
import type { LogEntry } from '@metamask/logger';
66
import { Logger } from '@metamask/logger';
7+
import { appendFileSync } from 'node:fs';
78
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
89
import { join } from 'node:path';
910

1011
import { getOcapHome } from '../ocap-home.ts';
1112
import { isProcessAlive } from '../utils.ts';
1213

14+
const ocapDir = getOcapHome();
15+
const logPath = join(ocapDir, 'daemon.log');
16+
const logger = new Logger({
17+
tags: ['daemon'],
18+
transports: [makeFileTransport(logPath)],
19+
});
20+
21+
// Install exit-cause handlers at module load, before main() runs, so
22+
// failures during kernel init also leave a fingerprint. daemon-entry
23+
// runs with `stdio: 'ignore'` under the CLI spawner (see
24+
// `daemon-spawn.ts`); without these, an uncaught exception, an
25+
// unhandled rejection, or a SIGHUP terminates the process silently
26+
// with no record in `daemon.log`. Silent deaths cost real debugging
27+
// time — see the run-notes for two past cases where a daemon
28+
// disappeared with no trace. Every terminating path now writes at
29+
// least one line before the process goes away.
30+
installFatalHandlers();
31+
1332
main().catch((error) => {
1433
process.stderr.write(`Daemon fatal: ${String(error)}\n`);
1534
process.exitCode = 1;
@@ -19,15 +38,8 @@ main().catch((error) => {
1938
* Main daemon entry point. Starts the daemon process and keeps it running.
2039
*/
2140
async function main(): Promise<void> {
22-
const ocapDir = getOcapHome();
2341
await mkdir(ocapDir, { recursive: true });
2442

25-
const logPath = join(ocapDir, 'daemon.log');
26-
const logger = new Logger({
27-
tags: ['daemon'],
28-
transports: [makeFileTransport(logPath)],
29-
});
30-
3143
const socketPath =
3244
process.env.OCAP_SOCKET_PATH ?? join(ocapDir, 'daemon.sock');
3345

@@ -129,15 +141,66 @@ async function readDaemonPid(pidPath: string): Promise<number | undefined> {
129141
/**
130142
* Create a file transport that writes logs to a file.
131143
*
132-
* @param logPath - The log file path.
144+
* @param logFilePath - The log file path.
133145
* @returns A log transport function.
134146
*/
135-
function makeFileTransport(logPath: string) {
136-
// eslint-disable-next-line @typescript-eslint/no-require-imports, n/global-require -- need sync fs for log transport
137-
const fs = require('node:fs') as typeof import('node:fs');
147+
function makeFileTransport(logFilePath: string) {
138148
return (entry: LogEntry): void => {
139149
const line = `[${new Date().toISOString()}] [${entry.level}] ${entry.message ?? ''} ${(entry.data ?? []).map(String).join(' ')}\n`;
140150
// eslint-disable-next-line n/no-sync -- synchronous write needed for log transport reliability
141-
fs.appendFileSync(logPath, line);
151+
appendFileSync(logFilePath, line);
142152
};
143153
}
154+
155+
/**
156+
* Install process-level handlers that guarantee a log line is
157+
* written for every terminating event before the daemon exits.
158+
*
159+
* The `@metamask/logger` dispatch routine is synchronous and the
160+
* file transport we're using here is `appendFileSync` under the
161+
* hood, so `logger.error(...)` from inside a fatal handler flushes
162+
* to disk before the process exits — no separate sync-write path
163+
* is required.
164+
*
165+
* Handlers registered:
166+
*
167+
* - `uncaughtException` — the classic silent-death path. Node's
168+
* default is to print the stack to stderr and exit with code 1;
169+
* under `stdio: 'ignore'` (how the daemon is spawned) that
170+
* default writes nowhere.
171+
* - `unhandledRejection` — currently defaults to a warning in
172+
* Node, but future Node versions treat it as uncaughtException;
173+
* either way we want a fingerprint.
174+
* - `SIGHUP` — sent when the controlling terminal disappears
175+
* (ssh session closed, laptop lid closed while the daemon was
176+
* under an interactive shell). Default action terminates the
177+
* process; installing a handler lets us log the fact before
178+
* exiting.
179+
* - `exit` — last-ditch record. Fires during every exit, including
180+
* the ones already logged by the handlers above.
181+
*/
182+
function installFatalHandlers(): void {
183+
/* eslint-disable n/no-process-exit -- fatal handlers must terminate deterministically */
184+
process.on('uncaughtException', (error: unknown) => {
185+
const detail =
186+
error instanceof Error ? (error.stack ?? error.message) : String(error);
187+
logger.error('Uncaught exception', detail);
188+
process.exit(1);
189+
});
190+
process.on('unhandledRejection', (reason: unknown) => {
191+
const detail =
192+
reason instanceof Error
193+
? (reason.stack ?? reason.message)
194+
: String(reason);
195+
logger.error('Unhandled rejection', detail);
196+
process.exit(1);
197+
});
198+
process.on('SIGHUP', () => {
199+
logger.error('SIGHUP received; exiting.');
200+
process.exit(0);
201+
});
202+
process.on('exit', (code) => {
203+
logger.error(`Process exiting (code=${code}).`);
204+
});
205+
/* eslint-enable n/no-process-exit */
206+
}

0 commit comments

Comments
 (0)