Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/kernel-cli/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed

- `kernel daemon start` refuses to start when another daemon is already listening on the same Unix socket, instead of unlinking the socket and orphaning the running process ([#952](https://github.com/MetaMask/ocap-kernel/pull/952))
- Daemon fatal-path visibility: `daemon-entry` now installs handlers for `uncaughtException`, `unhandledRejection`, `SIGHUP`, and `exit` that append a synchronous fingerprint line to `daemon.log` before terminating ([#966](https://github.com/MetaMask/ocap-kernel/pull/966))
- Without these, silent daemon deaths under `stdio: 'ignore'` (the CLI's default spawn mode) left no trace in the log; the operator saw only that the daemon was gone. Every terminating path now leaves at least one line.

## [0.1.0]

Expand Down
94 changes: 91 additions & 3 deletions packages/kernel-cli/src/commands/daemon-entry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,24 @@ import { startDaemon } from '@metamask/kernel-node-runtime/daemon';
import type { DaemonHandle } from '@metamask/kernel-node-runtime/daemon';
import type { LogEntry } from '@metamask/logger';
import { Logger } from '@metamask/logger';
import { appendFileSync } from 'node:fs';
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { join } from 'node:path';

import { getOcapHome } from '../ocap-home.ts';
import { isProcessAlive } from '../utils.ts';

// Install exit-cause handlers at module load, before main() runs, so
// failures during kernel init also leave a fingerprint. daemon-entry
// runs with `stdio: 'ignore'` under the CLI spawner (see
// `daemon-spawn.ts`); without these, an uncaught exception, an
// unhandled rejection, or a SIGHUP terminates the process silently
// with no record in `daemon.log`. Silent deaths cost real debugging
// time — see the run-notes for two past cases where a daemon
// disappeared with no trace. Every terminating path now writes at
// least one line before the process goes away.
installFatalHandlers(join(getOcapHome(), 'daemon.log'));

main().catch((error) => {
process.stderr.write(`Daemon fatal: ${String(error)}\n`);
process.exitCode = 1;
Expand Down Expand Up @@ -133,11 +145,87 @@ async function readDaemonPid(pidPath: string): Promise<number | undefined> {
* @returns A log transport function.
*/
function makeFileTransport(logPath: string) {
// eslint-disable-next-line @typescript-eslint/no-require-imports, n/global-require -- need sync fs for log transport
const fs = require('node:fs') as typeof import('node:fs');
return (entry: LogEntry): void => {
const line = `[${new Date().toISOString()}] [${entry.level}] ${entry.message ?? ''} ${(entry.data ?? []).map(String).join(' ')}\n`;
// eslint-disable-next-line n/no-sync -- synchronous write needed for log transport reliability
fs.appendFileSync(logPath, line);
appendFileSync(logPath, line);
};
}

/**
* Append a fatal-path entry to `daemon.log` synchronously. Used from
* `process.on('uncaughtException' | 'unhandledRejection' | 'SIGHUP')`
* handlers where the async logger pipeline can't be trusted to
* flush before the process exits. Best-effort: if the log file is
* unwritable we swallow the error rather than throw from a fatal
* handler.
*
* @param logPath - The daemon-log file path.
* @param message - Short label for the entry.
* @param detail - Optional extra data (stack, error, etc.) — coerced
* to string.
*/
function logFatalSync(
logPath: string,
message: string,
detail?: string | number,
): void {
try {
const tail = detail === undefined ? '' : ` ${detail}`;
const line = `[${new Date().toISOString()}] [error] ${message}${tail}\n`;
// eslint-disable-next-line n/no-sync -- fatal handler must flush before exit
appendFileSync(logPath, line);
} catch {
// Best-effort — the daemon is dying either way.
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/**
* Append a fatal-path entry to `daemon.log` synchronously. Used from
* `process.on('uncaughtException' | 'unhandledRejection' | 'SIGHUP')`
* handlers where the async logger pipeline can't be trusted to
* flush before the process exits. Best-effort: if the log file is
* unwritable we swallow the error rather than throw from a fatal
* handler.
*
* @param logPath - The daemon-log file path.
* @param message - Short label for the entry.
* @param detail - Optional extra data (stack, error, etc.) coerced
* to string.
*/
function logFatalSync(
logPath: string,
message: string,
detail?: string | number,
): void {
try {
const tail = detail === undefined ? '' : ` ${detail}`;
const line = `[${new Date().toISOString()}] [error] ${message}${tail}\n`;
// eslint-disable-next-line n/no-sync -- fatal handler must flush before exit
appendFileSync(logPath, line);
} catch {
// Best-effort — the daemon is dying either way.
}
}

The logger dispatch routine is already synchronous; async methods follow 'spray and pray' semantics, i.e. best effort.

I think the process.on handlers are doing the work here, not this function.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call — done in 70b96eb. Logger dispatch is indeed synchronous (Logger.#dispatch iterates transports via .forEach) and our file transport is appendFileSync, so logger.error(...) from a fatal handler flushes to disk before exit. Dropped the parallel sync helper.


/**
* Install process-level handlers that guarantee a log line is
* written for every terminating event before the daemon exits.
*
* Handlers registered:
*
* - `uncaughtException` — the classic silent-death path. Node's
* default is to print the stack to stderr and exit with code 1;
* under `stdio: 'ignore'` (how the daemon is spawned) that
* default writes nowhere.
* - `unhandledRejection` — currently defaults to a warning in
* Node, but future Node versions treat it as uncaughtException;
* either way we want a fingerprint.
* - `SIGHUP` — sent when the controlling terminal disappears
* (ssh session closed, laptop lid closed while the daemon was
* under an interactive shell). Default action terminates the
* process; installing a handler lets us log the fact before
* exiting.
* - `exit` — last-ditch record. Fires during every exit, including
* the ones already logged by the handlers above. Sync-safe: only
* sync APIs are usable here.
*
* @param logPath - The daemon-log file path.
*/
function installFatalHandlers(logPath: string): void {
/* eslint-disable n/no-sync, n/no-process-exit -- fatal handlers must flush synchronously and terminate deterministically */
process.on('uncaughtException', (error: unknown) => {
const detail =
error instanceof Error ? (error.stack ?? error.message) : String(error);
logFatalSync(logPath, 'Uncaught exception (about to exit):', detail);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logFatalSync(logPath, 'Uncaught exception (about to exit):', detail);
logger.error('Uncaught exception', detail);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — the handler now calls logger.error(...) directly, and the file-scoped logger it references is hoisted to module top level in the same commit.

process.exit(1);
});
process.on('unhandledRejection', (reason: unknown) => {
const detail =
reason instanceof Error
? (reason.stack ?? reason.message)
: String(reason);
logFatalSync(logPath, 'Unhandled rejection (about to exit):', detail);
process.exit(1);
});
process.on('SIGHUP', () => {
logFatalSync(logPath, 'SIGHUP received; exiting.');
process.exit(0);
});
process.on('exit', (code) => {
logFatalSync(logPath, `Process exiting (code=${code}).`);
});
/* eslint-enable n/no-sync, n/no-process-exit */
}
Loading