You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a client supplies a uuid for a multimodal item, it becomes the sole cache key with no binding to the payload: ProcessorInputs.get_mm_hashes appends the uuid verbatim (hashes.append(uuid_item) when no hash factors are in play, vllm/multimodal/processing/inputs.py), and both cache layers key on it — the P0 processor cache (MultiModalProcessorOnlyCache.get_and_update_item) and the engine-side encoder-output cache (free_encoder_mm_hashes). Nothing verifies that a uuid maps to the same content it was first cached under. Verified on main @ f4eccda (the hash-factor scoping from #54918 / #7292ee279 does not bind content when no processor kwargs are in play).
Two failure classes follow from one colliding uuid (same id, different bytes):
Different processed lengths → fatal engine crash. A later request splices placeholders from its own processing but is served the stale cached features; the merge mismatches and V1 kills the entire engine tree ([Bug]: One bad multimodal item (feature/placeholder mismatch) kills the entire V1 engine tree instead of failing that request #55546). Observed: a 10 s clip was processed and cached under audio-2-1-0; a later 30 s request reused that uuid and was served 138-token features against its 638-placeholder splice — the whole DP8 deployment died.
Same processed length, different content → silent wrong-answer serving. The merge succeeds; the model consumes the stale item's features under the fresh request's splice. No error anywhere; outputs are simply wrong. This is the more dangerous class and it leaves no trace.
To Reproduce
Serve any model with a multimodal audio processor (P0 processor cache enabled; processor-only mode under internal-LB DP suffices).
Send a request with an audio item carrying uuid: "item-1" and payload A; let it be processed and cached.
Send another request with uuid: "item-1" and payload B of a different duration, routed to the same API server (the cache is per-process).
We triggered this in production-shaped benchmarking when two tool paths minted uuids from overlapping id spaces for different-length slices of the same clip — realistic whenever more than one service mints item ids.
Expected behavior
At minimum, prevent inconsistent serving from a uuid collision: fold cheap content metadata into the key (e.g., a short digest of uuid || payload byte length || sample count) so two payloads that process differently can never share an entry, and evict the offending mm_hash on any merge failure. Stronger options: a debug/audit mode that spot-checks cached features against fresh processing, or documenting loudly that the uuid is not just a performance hint but a content-stability contract the server cannot check — gateway id spaces are exactly the kind of thing that collides in practice.
Environment
vLLM: v0.26.1rc1.dev1246+g1a085dadf; verbatim-uuid keying re-verified on main @ f4eccda
Describe the bug
When a client supplies a
uuidfor a multimodal item, it becomes the sole cache key with no binding to the payload:ProcessorInputs.get_mm_hashesappends the uuid verbatim (hashes.append(uuid_item)when no hash factors are in play,vllm/multimodal/processing/inputs.py), and both cache layers key on it — the P0 processor cache (MultiModalProcessorOnlyCache.get_and_update_item) and the engine-side encoder-output cache (free_encoder_mm_hashes). Nothing verifies that a uuid maps to the same content it was first cached under. Verified onmain@ f4eccda (the hash-factor scoping from #54918 / #7292ee279 does not bind content when no processor kwargs are in play).Two failure classes follow from one colliding uuid (same id, different bytes):
audio-2-1-0; a later 30 s request reused that uuid and was served 138-token features against its 638-placeholder splice — the whole DP8 deployment died.To Reproduce
uuid: "item-1"and payload A; let it be processed and cached.uuid: "item-1"and payload B of a different duration, routed to the same API server (the cache is per-process).We triggered this in production-shaped benchmarking when two tool paths minted uuids from overlapping id spaces for different-length slices of the same clip — realistic whenever more than one service mints item ids.
Expected behavior
At minimum, prevent inconsistent serving from a uuid collision: fold cheap content metadata into the key (e.g., a short digest of
uuid || payload byte length || sample count) so two payloads that process differently can never share an entry, and evict the offendingmm_hashon any merge failure. Stronger options: a debug/audit mode that spot-checks cached features against fresh processing, or documenting loudly that the uuid is not just a performance hint but a content-stability contract the server cannot check — gateway id spaces are exactly the kind of thing that collides in practice.Environment
main@ f4eccdaGemma4UnifiedAudioFeatureExtractor), DP8 internal LB, 8x B200