Commit 4cddd8b
committed
fix: per-write deadline on listener + per-read deadline on source; ReadHeaderTimeout
Recurring symptom: HTTPS to the server stops responding after some
time even though the process is alive — TLS handshake completes
(verified with openssl s_client) but the HTTP layer hangs and curl
returns 'i/o timeout' for every request, stream and metadata POST
alike. CLOSE_WAIT socket count climbs into the dozens.
Root cause: handler-side I/O has no deadline.
- serveStreamData's out.Write / w.Write to a slow or
half-disconnected listener blocks indefinitely (TCP send buffer
full, never drains). The handler goroutine sits there, the
listener slot stays registered, the conn ends up in CLOSE_WAIT,
and over time enough of these accumulate that the runtime can
no longer service new HTTPS requests in a timely way.
- handleSource's bufrw.Read on a hijacked conn has the same
problem in the other direction: a source that drops without a
FIN (network blip, encoder crash) wedges the handler.
The fix is the standard Go-network-server defence:
- Listener: http.NewResponseController(w).SetWriteDeadline before
every Write. 30 s — typical Write to a healthy listener
finishes in <1 ms; this only fires when the receiver is gone.
- Source: conn.SetReadDeadline(60 s) refreshed after every
successful read. A continuously-streaming source never sees it;
a silent connection dies cleanly.
Also: switched both http.Server's ReadTimeout (10 s, applies to the
WHOLE request including streaming bodies, wrong for a server that
ingests source audio over PUT/SOURCE) to ReadHeaderTimeout. Same
slowloris protection without penalising long-running uploads.1 parent 2bcb8e7 commit 4cddd8b
2 files changed
Lines changed: 45 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
155 | 164 | | |
156 | 165 | | |
157 | 166 | | |
| |||
178 | 187 | | |
179 | 188 | | |
180 | 189 | | |
| 190 | + | |
181 | 191 | | |
182 | 192 | | |
183 | 193 | | |
| |||
504 | 514 | | |
505 | 515 | | |
506 | 516 | | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
507 | 533 | | |
508 | 534 | | |
509 | 535 | | |
| |||
573 | 599 | | |
574 | 600 | | |
575 | 601 | | |
| 602 | + | |
576 | 603 | | |
577 | 604 | | |
578 | 605 | | |
| |||
592 | 619 | | |
593 | 620 | | |
594 | 621 | | |
| 622 | + | |
595 | 623 | | |
596 | 624 | | |
597 | 625 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
280 | 280 | | |
281 | 281 | | |
282 | 282 | | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
288 | 295 | | |
289 | 296 | | |
290 | 297 | | |
| |||
312 | 319 | | |
313 | 320 | | |
314 | 321 | | |
315 | | - | |
316 | | - | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
317 | 327 | | |
318 | 328 | | |
319 | 329 | | |
| |||
0 commit comments