Memory Management Tracklist

[severfire]

Items to Investigate

The following optimizations should be checked and investigated for potential memory savings:

1. jemalloc via LD_PRELOAD + MALLOC_CONF

   • Use jemalloc as a drop-in replacement for the system allocator
   • Configure memory allocation parameters via MALLOC_CONF

2. Garbage collection tuning

   • Add gc.set_threshold(0, 0, 0) in master process tail
   • Add second gc.freeze() call in worker processes

3. Preload refactoring

   • Move setup_llm_caching initialization into _run_master_preload
   • Move Sentry initialization into _run_master_preload

prestodb/presto#8993
https://instagram-engineering.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf

related merged PRs:
#12587
#12364
#12313

open PRs:
#12778

related:
#12588

[severfire]

Issues on growing memory. Tested 60 running workers with 400 concurrent requests on small flow.

[ 1s] RAM 8.93 GB done=0/400 ok=0 err=0
[ 2s] RAM 9.37 GB done=0/400 ok=0 err=0
[ 3s] RAM 9.90 GB done=0/400 ok=0 err=0
[ 4s] RAM 10.28 GB done=4/400 ok=4 err=0
[ 5s] RAM 10.42 GB done=14/400 ok=14 err=0
[ 6s] RAM 10.55 GB done=24/400 ok=24 err=0
[ 7s] RAM 10.65 GB done=36/400 ok=36 err=0
[ 8s] RAM 10.71 GB done=46/400 ok=46 err=0
[ 9s] RAM 10.74 GB done=56/400 ok=56 err=0
[ 10s] RAM 10.77 GB done=65/400 ok=65 err=0
[ 11s] RAM 10.82 GB done=74/400 ok=74 err=0
[ 12s] RAM 10.85 GB done=89/400 ok=89 err=0
[ 13s] RAM 10.89 GB done=104/400 ok=104 err=0
[ 14s] RAM 10.89 GB done=113/400 ok=113 err=0
[ 15s] RAM 10.93 GB done=121/400 ok=121 err=0
[ 16s] RAM 10.95 GB done=133/400 ok=133 err=0
[ 17s] RAM 10.98 GB done=140/400 ok=140 err=0
[ 18s] RAM 11.02 GB done=149/400 ok=149 err=0
[ 19s] RAM 11.03 GB done=160/400 ok=160 err=0
[ 20s] RAM 11.05 GB done=169/400 ok=169 err=0
[ 21s] RAM 11.01 GB done=179/400 ok=179 err=0
[ 22s] RAM 11.08 GB done=193/400 ok=193 err=0
[ 23s] RAM 11.05 GB done=201/400 ok=201 err=0
[ 24s] RAM 11.13 GB done=210/400 ok=210 err=0
[ 25s] RAM 11.12 GB done=221/400 ok=221 err=0
[ 26s] RAM 11.14 GB done=228/400 ok=228 err=0
[ 27s] RAM 11.16 GB done=238/400 ok=238 err=0
[ 28s] RAM 11.20 GB done=248/400 ok=248 err=0
[ 29s] RAM 11.16 GB done=257/400 ok=257 err=0
[ 30s] RAM 11.17 GB done=267/400 ok=267 err=0
[ 31s] RAM 11.18 GB done=276/400 ok=276 err=0
[ 32s] RAM 11.22 GB done=287/400 ok=287 err=0
[ 33s] RAM 11.22 GB done=292/400 ok=292 err=0
[ 34s] RAM 11.24 GB done=308/400 ok=308 err=0
[ 35s] RAM 11.21 GB done=316/400 ok=316 err=0
[ 36s] RAM 11.25 GB done=324/400 ok=324 err=0
[ 37s] RAM 11.26 GB done=332/400 ok=332 err=0
[ 38s] RAM 11.27 GB done=339/400 ok=339 err=0
[ 39s] RAM 11.27 GB done=351/400 ok=351 err=0
[ 40s] RAM 11.30 GB done=364/400 ok=364 err=0
[ 41s] RAM 11.32 GB done=382/400 ok=382 err=0
[ 42s] RAM 11.31 GB done=393/400 ok=393 err=0
[ 43s] RAM 11.32 GB done=399/400 ok=399 err=0
[ 44s] RAM 11.32 GB done=399/400 ok=399 err=0