Skip to content

Bug: Firecracker VMM process deadlocks/hangs on SIGTERM during graceful shutdown #789

Description

@Nachiket-Roy

Description

When a container is gracefully stopped or killed in Kubernetes, containerd sends a SIGTERM (15) to the container's init process (the VMM process, such as Firecracker).

According to Firecracker's documentation, its signal handlers are not async-signal-safe because they perform lock-based flushes of logging and metrics buffers. If a signal interrupts a thread that already holds one of these locks, it results in a deadlock, leaving the Firecracker process unresponsive and hanging on host signals.

Because it hangs, it does not terminate until the Kubernetes termination grace period (default 30s) expires and containerd sends a SIGKILL (9). Since Firecracker doesn't support ACPI guest shutdowns from signals anyway, sending SIGTERM abruptly stops the VM just like SIGKILL, but with a high risk of deadlock.

Steps to Reproduce

  1. Start a urunc container using the firecracker hypervisor.
  2. Signal the container gracefully:
urunc kill <container-id> 15
  1. Observe that the Firecracker process remains alive in the process table, unresponsive to SIGTERM, until forced-killed with SIGKILL.

Expected Behavior

The Firecracker process should terminate immediately when signaled to stop, allowing the pod to be cleaned up without waiting for the grace period.

Suggested Fix

Map unix.SIGTERM to unix.SIGKILL in the Firecracker Signal method to force immediate termination and bypass the unsafe signal handler paths:

func (fc *Firecracker) Signal(pid int, signal unix.Signal) error {
	if signal == unix.SIGTERM {
		return unix.Kill(pid, unix.SIGKILL)
	}
	return unix.Kill(pid, signal)
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions