Skip to content

[Bug]: Client-supplied mm uuid is the sole cache key with no content binding - stale serving or fatal engine crash on collision #55547

Description

@AbroadConfirm

Describe the bug

When a client supplies a uuid for a multimodal item, it becomes the sole cache key with no binding to the payload: ProcessorInputs.get_mm_hashes appends the uuid verbatim (hashes.append(uuid_item) when no hash factors are in play, vllm/multimodal/processing/inputs.py), and both cache layers key on it — the P0 processor cache (MultiModalProcessorOnlyCache.get_and_update_item) and the engine-side encoder-output cache (free_encoder_mm_hashes). Nothing verifies that a uuid maps to the same content it was first cached under. Verified on main @ f4eccda (the hash-factor scoping from #54918 / #7292ee279 does not bind content when no processor kwargs are in play).

Two failure classes follow from one colliding uuid (same id, different bytes):

  1. Different processed lengths → fatal engine crash. A later request splices placeholders from its own processing but is served the stale cached features; the merge mismatches and V1 kills the entire engine tree ([Bug]: One bad multimodal item (feature/placeholder mismatch) kills the entire V1 engine tree instead of failing that request #55546). Observed: a 10 s clip was processed and cached under audio-2-1-0; a later 30 s request reused that uuid and was served 138-token features against its 638-placeholder splice — the whole DP8 deployment died.
  2. Same processed length, different content → silent wrong-answer serving. The merge succeeds; the model consumes the stale item's features under the fresh request's splice. No error anywhere; outputs are simply wrong. This is the more dangerous class and it leaves no trace.

To Reproduce

  1. Serve any model with a multimodal audio processor (P0 processor cache enabled; processor-only mode under internal-LB DP suffices).
  2. Send a request with an audio item carrying uuid: "item-1" and payload A; let it be processed and cached.
  3. Send another request with uuid: "item-1" and payload B of a different duration, routed to the same API server (the cache is per-process).
  4. The second request fails the merge and the engine tree dies (see [Bug]: One bad multimodal item (feature/placeholder mismatch) kills the entire V1 engine tree instead of failing that request #55546). With equal durations instead, it silently answers using payload A's features.

We triggered this in production-shaped benchmarking when two tool paths minted uuids from overlapping id spaces for different-length slices of the same clip — realistic whenever more than one service mints item ids.

Expected behavior

At minimum, prevent inconsistent serving from a uuid collision: fold cheap content metadata into the key (e.g., a short digest of uuid || payload byte length || sample count) so two payloads that process differently can never share an entry, and evict the offending mm_hash on any merge failure. Stronger options: a debug/audit mode that spot-checks cached features against fresh processing, or documenting loudly that the uuid is not just a performance hint but a content-stability contract the server cannot check — gateway id spaces are exactly the kind of thing that collides in practice.

Environment

  • vLLM: v0.26.1rc1.dev1246+g1a085dadf; verbatim-uuid keying re-verified on main @ f4eccda
  • Gemma 4 audio (Gemma4UnifiedAudioFeatureExtractor), DP8 internal LB, 8x B200

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions