Summary
On vLLM 0.24.0 (V1, default multiprocess), the EngineCore child performs two consecutive full-heap generation-2 GC passes during SIGTERM teardown, costing ~3.9–6.6 s to collect a few hundred objects in a process that is about to exit. Every offline LLM teardown pays this (benchmark iterations, test suites, repeated engine lifecycles).
Measurements
Observed via VLLM_GC_DEBUG='{"top_objects":20}' on two hosts, bare workload (Qwen/Qwen2.5-7B-Instruct, batch 32, greedy, no structured outputs), immediately after the shutdown sequence logs ([shutdown] EngineCore: trigger received signal=SIGTERM → exiting busy loop):
1×A100-SXM4 40GB (Ubuntu 24.04, 30 vCPU):
(EngineCore pid=3684) GC took 4442.360ms to complete. Collected 377 objects (out of 1052645) in GC generation 2.
(EngineCore pid=3684) GC took 2164.036ms to complete. Collected 377 objects (out of 1035628) in GC generation 2.
1×H100 PCIe 80GB (Ubuntu 24.04, 26 vCPU):
GC took 2738.397ms to complete. Collected 377 objects (out of 1095452) in GC generation 2.
GC took 1391.236ms to complete. Collected 377 objects (out of 1078431) in GC generation 2.
Top collected-object classes are the startup graph, not request garbage: function (~150k), tuple, cell, dict, weakref.ReferenceType, vllm.v1.core.kv_cache_utils.KVCacheBlock (~23k).
During serving the same process's gen-2 tracked set is only ~24k–64k objects (freeze_gc_heap() working as intended); the ~1M-object heap becomes visible to the collector only on the shutdown path — consistent with gc.unfreeze() in EngineCore.shutdown() followed by full collection(s).
Why it may be worth changing
The process is exiting: OS teardown reclaims everything regardless, so 4–7 s of full-heap GC buys nothing on this path. Skipping the post-unfreeze full collections when the process is terminating (or leaving the heap frozen on exit paths) would make offline engine teardown ~seconds faster with no behavioral difference.
Repro
VLLM_GC_DEBUG='{"top_objects":20}' python -c "
from vllm import LLM, SamplingParams
llm = LLM(model='Qwen/Qwen2.5-7B-Instruct', max_model_len=2048)
llm.generate(['hello'], SamplingParams(max_tokens=16))
"
# watch the EngineCore GC lines after the SIGTERM shutdown sequenceContext: found while running the GC diagnostics for #48229 (multiprocess frozen-step investigation); split out per discussion there.
Summary
On vLLM 0.24.0 (V1, default multiprocess), the
EngineCorechild performs two consecutive full-heap generation-2 GC passes during SIGTERM teardown, costing ~3.9–6.6 s to collect a few hundred objects in a process that is about to exit. Every offlineLLMteardown pays this (benchmark iterations, test suites, repeated engine lifecycles).Measurements
Observed via
VLLM_GC_DEBUG='{"top_objects":20}'on two hosts, bare workload (Qwen/Qwen2.5-7B-Instruct, batch 32, greedy, no structured outputs), immediately after the shutdown sequence logs ([shutdown] EngineCore: trigger received signal=SIGTERM→exiting busy loop):1×A100-SXM4 40GB (Ubuntu 24.04, 30 vCPU):
1×H100 PCIe 80GB (Ubuntu 24.04, 26 vCPU):
Top collected-object classes are the startup graph, not request garbage:
function(~150k),tuple,cell,dict,weakref.ReferenceType,vllm.v1.core.kv_cache_utils.KVCacheBlock(~23k).During serving the same process's gen-2 tracked set is only ~24k–64k objects (
freeze_gc_heap()working as intended); the ~1M-object heap becomes visible to the collector only on the shutdown path — consistent withgc.unfreeze()inEngineCore.shutdown()followed by full collection(s).Why it may be worth changing
The process is exiting: OS teardown reclaims everything regardless, so 4–7 s of full-heap GC buys nothing on this path. Skipping the post-unfreeze full collections when the process is terminating (or leaving the heap frozen on exit paths) would make offline engine teardown ~seconds faster with no behavioral difference.
Repro
Context: found while running the GC diagnostics for #48229 (multiprocess frozen-step investigation); split out per discussion there.