Commit b127147
committed
fix: create the RequestExecutingAgent ProcessPool in initialize
ProcessPool was built lazily on the first processPool() call, which happens
inside execute(), after getBulkRequests() has already loaded a whole
BulkRequest into memory. All MinProcess workers are forked at that instant,
and because a worker never unwinds the frame it was forked from, every one
of them pins that cycle's Requests for the lifetime of the agent.
A core dump of one worker from a 60-process pool running BulkRequest=3000
still held the parent's requestsToExecute list and getRequests dict:
3003 Request, 3764 Operation and 40973 File objects, 54.8 MiB of live data
in a process that only ever handles one request at a time.
Creating the pool at the end of initialize() forks from the main thread
before any request is fetched, and keeps the RequestDB engine out of the
workers as well.
Claude-Session: https://claude.ai/code/session_01Wj93RvpYkvmL2AGmUHneRN1 parent aaa32ed commit b127147
1 file changed
Lines changed: 15 additions & 1 deletion
Lines changed: 15 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
118 | | - | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
119 | 124 | | |
120 | 125 | | |
121 | 126 | | |
| |||
273 | 278 | | |
274 | 279 | | |
275 | 280 | | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
276 | 290 | | |
277 | 291 | | |
278 | 292 | | |
| |||
0 commit comments