Skip to content

fix(liveness): skip self.close() on SIGTERM during HMR/watch - #1212

Open
guitavano wants to merge 1 commit into
mainfrom
fix/hmr-sigterm-self-close
Open

fix(liveness): skip self.close() on SIGTERM during HMR/watch#1212
guitavano wants to merge 1 commit into
mainfrom
fix/hmr-sigterm-self-close

Conversation

@guitavano

@guitavano guitavano commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Skip self.close() in the liveness middleware's SIGTERM handler when running in dev mode (--unstable-hmr or --watch)
  • In production (k8s), SIGTERM comes from kubelet and the process must shut down — behavior unchanged
  • In local dev, Deno's HMR/watch sends SIGTERM to restart the child process — self.close() was killing it before the restart cycle could complete, breaking hot-reload entirely

Root cause

Deno's HMR uses a parent/child model: parent watches files → sends SIGTERM to child → child exits → parent relaunches. The self.close() call in the SIGTERM handler was terminating the Deno runtime before HMR could complete its restart, causing every dev server to die on the first file change.

Test plan

  • Run deno task dev on any deco site — verify HMR restarts correctly on file changes instead of dying
  • Run deno run -A --watch main.ts — verify watch mode restarts correctly
  • Verify production k8s liveness probe behavior is unchanged (SIGTERM still calls self.close() when not in dev mode)

🤖 Generated with Claude Code


Summary by cubic

Skip calling self.close() on SIGTERM outside Kubernetes so Deno HMR/--watch can restart cleanly; production (k8s) shutdown behavior is unchanged.

  • Bug Fixes
    • Detect k8s via KUBERNETES_SERVICE_HOST; only call self.close() when present.
    • Restore hot-reload by letting Deno HMR/--watch restart the process without an early runtime close.

Written for commit e8a32a1. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Bug Fixes
    • Improved SIGTERM shutdown handling on non-Windows platforms to avoid disrupting development restarts (e.g., hot reload/watch workflows).
    • Liveness checks continue to run and log results during SIGTERM.
    • The service now only triggers a full close when running in Kubernetes, while non-Kubernetes environments keep running to support development recovery behavior.

@github-actions

Copy link
Copy Markdown
Contributor

Tagging Options

Should a new tag be published when this PR is merged?

  • 👍 for Patch 1.202.1 update
  • 🎉 for Minor 1.203.0 update
  • 🚀 for Major 2.0.0 update

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The liveness middleware still runs checks and logs on SIGTERM, but it now calls self.close() only when Kubernetes is detected through KUBERNETES_SERVICE_HOST.

Changes

SIGTERM handling

Layer / File(s) Summary
Kubernetes shutdown guard
runtime/middlewares/liveness.ts
The SIGTERM listener derives isK8s from KUBERNETES_SERVICE_HOST and only calls self.close() when that flag is true.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

A bunny heard SIGTERM at play,
But closed the burrow only Kubernetes way.
Checks still sparkle, logs still gleam,
Yet local hops stay live in the stream. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: skipping self.close() on SIGTERM during HMR/watch.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/hmr-sigterm-self-close

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@runtime/middlewares/liveness.ts`:
- Around line 72-74: The dev-mode check in liveness handling is reading Deno CLI
flags from Deno.args, but those flags are not available there, so the shutdown
path still runs in development. Update the isHMR/isWatch/isDev logic in
runtime/middlewares/liveness.ts to use a launcher-provided environment variable
instead of Deno.args, and keep the SIGTERM/self.close() behavior gated off when
that dev-mode variable is set.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8057ab88-95a0-4bc6-9650-602eb5449d21

📥 Commits

Reviewing files that changed from the base of the PR and between ce2cbd7 and 52cfcb3.

📒 Files selected for processing (1)
  • runtime/middlewares/liveness.ts

Comment thread runtime/middlewares/liveness.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread runtime/middlewares/liveness.ts Outdated
Deno's HMR and --watch modes use a parent/child process model where the
parent sends SIGTERM to signal the child to restart. The liveness
middleware's SIGTERM handler was calling self.close(), which kills the
process before Deno can complete its restart cycle — causing every dev
server to die on the first file change instead of hot-reloading.

Only call self.close() in k8s (detected via KUBERNETES_SERVICE_HOST)
where SIGTERM means the pod must shut down. Deno runtime flags like
--unstable-hmr and --watch are not visible in Deno.args, so detecting
dev mode directly is not reliable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@guitavano
guitavano force-pushed the fix/hmr-sigterm-self-close branch from 52cfcb3 to e8a32a1 Compare June 25, 2026 19:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@runtime/middlewares/liveness.ts`:
- Around line 75-82: The SIGTERM handler in liveness.ts now gates self.close()
on Kubernetes detection, which makes non-K8s production skips the default
shutdown path. Update the logic in the SIGTERM listener so the exception is
based on the specific dev/watch/HMR mode instead of isK8s, and keep self.close()
as the default behavior for normal shutdown handling in
runChecks()/Deno.addSignalListener.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8e339e50-1a89-43a3-aceb-7a1bc0701a13

📥 Commits

Reviewing files that changed from the base of the PR and between 52cfcb3 and e8a32a1.

📒 Files selected for processing (1)
  • runtime/middlewares/liveness.ts

Comment on lines +75 to +82
const isK8s = Boolean(Deno.env.get("KUBERNETES_SERVICE_HOST"));
try {
if (Deno.build.os !== "windows") {
Deno.addSignalListener("SIGTERM", () => {
const checks = runChecks();
console.log(checks);
self.close();
if (isK8s) {
self.close();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Don't use Kubernetes presence as the shutdown default.

Lines 75-82 now change SIGTERM behavior from “close unless we're in watch/HMR” to “close only in Kubernetes”. That widens the behavior change to every non-k8s runtime, so ordinary production deployments outside Kubernetes will now skip self.close() too. The exception should be keyed off the specific dev/watch mode, with self.close() remaining the default for normal SIGTERM handling.

Suggested change
-  const isK8s = Boolean(Deno.env.get("KUBERNETES_SERVICE_HOST"));
+  const isWatchMode = Deno.env.get("DECO_WATCH_MODE") === "1";
   try {
     if (Deno.build.os !== "windows") {
       Deno.addSignalListener("SIGTERM", () => {
         const checks = runChecks();
         console.log(checks);
-        if (isK8s) {
+        if (!isWatchMode) {
           self.close();
         }
       });
     }
   } catch (err) {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const isK8s = Boolean(Deno.env.get("KUBERNETES_SERVICE_HOST"));
try {
if (Deno.build.os !== "windows") {
Deno.addSignalListener("SIGTERM", () => {
const checks = runChecks();
console.log(checks);
self.close();
if (isK8s) {
self.close();
const isWatchMode = Deno.env.get("DECO_WATCH_MODE") === "1";
try {
if (Deno.build.os !== "windows") {
Deno.addSignalListener("SIGTERM", () => {
const checks = runChecks();
console.log(checks);
if (!isWatchMode) {
self.close();
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@runtime/middlewares/liveness.ts` around lines 75 - 82, The SIGTERM handler in
liveness.ts now gates self.close() on Kubernetes detection, which makes non-K8s
production skips the default shutdown path. Update the logic in the SIGTERM
listener so the exception is based on the specific dev/watch/HMR mode instead of
isK8s, and keep self.close() as the default behavior for normal shutdown
handling in runChecks()/Deno.addSignalListener.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant