Description:
soft serve spawns git child processes to handle operations but doesn't reap them, causing zombie processes to accumulate until the host runs out of PIDs or fork/exec fails.
Reproduction:
- Run soft serve on any machine (container or bare metal)
- Perform git operations against it (clone, push, pull via SSH or git protocol)
- Observe zombie processes accumulating under the soft parent:
# ps aux | awk '$8 ~ /Z/'
root 2592996 ... [git] <defunct>
root 2593008 ... [git] <defunct>
root 2593023 ... [git] <defunct>
...
After enough operations, PID exhaustion occurs:
fork/exec /proc/self/fd/6: resource temporarily unavailable
Root cause:
The Go process does not appear to handle SIGCHLD or call waitpid()/wait4() on its child processes. Sending SIGCHLD to the soft process manually reaps all zombies immediately.
kill -s SIGCHLD $(pgrep soft)
Impact:
- In Kubernetes, this made nodes unusable within hours (30k+ zombies in under 24h)
- All pods on the affected node fail with CreateContainerError: resource temporarily unavailable
- The only recovery was a full node reboot
Expected behavior:
Child git processes should be reaped automatically — either via a SIGCHLD handler with waitpid(-1, ...) in a loop, or by running the process with an init supervisor like tini or catatonio that handles reaping.
Environment:
- charmcli/soft-serve:latest (image sha256:7742f301ab4e5c24eccf999b10aabb97ec64e18c23f693a16599071d49ae5bf4)
- Running inside a Kubernetes container (containerd 1.7.30)
Description:
soft serve spawns git child processes to handle operations but doesn't reap them, causing zombie processes to accumulate until the host runs out of PIDs or fork/exec fails.
Reproduction:
After enough operations, PID exhaustion occurs:
fork/exec /proc/self/fd/6: resource temporarily unavailable
Root cause:
The Go process does not appear to handle SIGCHLD or call waitpid()/wait4() on its child processes. Sending SIGCHLD to the soft process manually reaps all zombies immediately.
kill -s SIGCHLD $(pgrep soft)
Impact:
Expected behavior:
Child git processes should be reaped automatically — either via a SIGCHLD handler with waitpid(-1, ...) in a loop, or by running the process with an init supervisor like tini or catatonio that handles reaping.
Environment: