Skip to content

Commit c43d1c6

Browse files
committed
feat(runtime): document FastCGI keepalive starving php-fpm
A php-fpm child stays bound to its FastCGI connection for as long as that connection lives, so an nginx keepalive pool to php-fpm parks workers rather than idle sockets. Sized at or above `pm.max_children` it can pin every child while arriving requests wait for one to come free. What makes it expensive to diagnose is that every obvious instrument says "fine": the container sits at 0 % CPU with flat memory, the php-fpm slowlog stays empty at a 5 s threshold because the scripts were never slow, and the stalled requests answer HTTP 200 — after a minute. The reference states the full signature, since any one of those readings alone points somewhere else. It also hides from local testing: only a client that issues everything at once builds the queue, so it appears over HTTP/2 through a reverse proxy and not when the application container is addressed directly. Measured on a TYPO3 backend behind Caddy with pm.max_children = 10: 61005 ms slowest request with `keepalive 16`, 19614 ms with 8, 107 ms with the pool removed. Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>
1 parent c8867db commit c43d1c6

2 files changed

Lines changed: 60 additions & 0 deletions

File tree

skills/docker-development/SKILL.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,3 +134,4 @@ Exclude: `.git`, `node_modules`/`vendor`, `.env*`, `*.pem`, `*.key`
134134
- `references/bind-mount-ownership.md` -- root-owned bind-mount artifacts
135135
- `references/gpg-verification.md` -- gpgv patterns; stale keybox locks
136136
- `references/build-secret-leaks.md` -- `ARG` lands in SLSA provenance
137+
- `references/php-fpm-worker-starvation.md` -- FastCGI keepalive pins workers
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# nginx FastCGI keepalive starves php-fpm
2+
3+
A php-fpm child stays bound to its FastCGI connection for as long as that
4+
connection lives. An nginx keepalive pool to php-fpm therefore does not park
5+
idle sockets — it parks **workers**. Size the pool at or above
6+
`pm.max_children` and it can pin every child, leaving arriving requests to wait
7+
for one to come free.
8+
9+
```nginx
10+
upstream php-fpm {
11+
server 127.0.0.1:9000;
12+
keepalive 16; # against pm.max_children = 10
13+
}
14+
location ~ \.php$ {
15+
fastcgi_keep_conn on; # <- makes the pool real
16+
}
17+
```
18+
19+
## The signature — all of it at once, or it is something else
20+
21+
- container at **0 % CPU**, memory flat — nothing is working
22+
- php-fpm **slowlog empty** at a low threshold
23+
(`request_slowlog_timeout = 5s`): the scripts were never slow, they never got
24+
a worker
25+
- the stalled requests answer **HTTP 200 after ~60 s**, with TTFB equal to the
26+
total, and the edge proxy's access log reports the same duration, so the wait
27+
is behind it
28+
- the very same URLs replayed **one at a time** in the same session take
29+
~100 ms
30+
31+
The empty slowlog is the decisive one: it separates "slow code" from "never
32+
scheduled", and it is the measurement most likely to be skipped.
33+
34+
## Why it hides from local testing
35+
36+
Only a client that can issue everything at once builds the queue. Over HTTP/2
37+
through a reverse proxy the page's requests share one connection and arrive
38+
together; a direct HTTP/1.1 client opens at most six connections per origin and
39+
never triggers it. Reproduce **with the proxy in the path** — a stack tested by
40+
addressing the application container directly measures a load shape that
41+
production never sees.
42+
43+
## Measured
44+
45+
TYPO3 backend behind Caddy, `pm.max_children = 10`, opening the backend shell:
46+
47+
| nginx | slowest request | over 5 s |
48+
|---|---|---|
49+
| `keepalive 16` + `fastcgi_keep_conn on` | 61005 ms | 5–8 of 24 |
50+
| `keepalive 8` + `fastcgi_keep_conn on` | 19614 ms | 1 of 24 |
51+
| no pool, `fastcgi_keep_conn off` | **107 ms** | none |
52+
53+
The intermediate value still costs 19.6 s, so this is not a tuning question:
54+
remove the pool. `fastcgi_keep_conn off` is the nginx default, and connection
55+
setup to `127.0.0.1` is not worth holding a worker for.
56+
57+
Raising `pm.max_children` instead trades memory for the same failure one load
58+
step further out — and where the container has a memory cap, the headroom for
59+
`keepalive`-many concurrently pinned children is not there to give.

0 commit comments

Comments
 (0)