fix: Make TokenizedPrompt.MultiModalFeatures per-prompt

[albertoperdomo2]

What type of PR is this?

/kind bug
/kind feature

What this PR does / why we need it:

What this PR does

Introduces a PromptTokens struct that bundles token IDs and multimodal features per prompt, replacing the two parallel slices on TokenizedPrompt (PerPromptTokens [][]uint32 and MultiModalFeatures []MultiModalFeature). Renames TokenizedPrompt to TokenizedRequest to distinguish the request-level container from the new per-prompt type.

Before:

type TokenizedPrompt struct {
    PerPromptTokens    [][]uint32
    MultiModalFeatures []MultiModalFeature  // flat, wrong
    CacheSalt          string
}

After:

type TokenizedRequest struct {
    Prompts   []PromptTokens
    CacheSalt string
}

type PromptTokens struct {
    TokenIDs           []uint32
    MultiModalFeatures []MultiModalFeature
}

The association between tokens and their multimodal features is now structural, not positional. Consumers no longer need bounds checks or guards like if len(tp.PerPromptTokens) == 1 to pair features with the correct token array.

The InferenceRequestBody field is renamed from TokenizedPrompt to TokenizedRequest to match the type. The data-key identifiers (TokenizedPromptDataKey, tokenizedPromptKeyID) are intentionally unchanged since they are plugin registry keys, not type names.

Consumers updated:

• blockkeys.go: Iterates tp.Prompts directly; the len(tp.PerPromptTokens) == 1 guard disappears because each PromptTokens carries its own features.
• multimodal/producer.go (ExtractMMItems): Iterates tp.Prompts accessing p.MultiModalFeatures.
• disagg/multimodal_helpers.go (hasMultimodalContent): Checks len(p.MultiModalFeatures) > 0 per prompt.
• approximateprefix/hashing.go: Iterates tp.Prompts using p.TokenIDs.
• vllmgrpc.go: Constructs TokenizedRequest with Prompts: []PromptTokens{{...}} for pre-tokenized gRPC inputs.

Why we need it

PerPromptTokens was [][]uint32 (per-prompt) but MultiModalFeatures was []MultiModalFeature (flat). MM feature offsets are only valid relative to one token array, but the flat shape forced consumers to assume which prompt the features belonged to. The blockkeys.go consumer had an explicit len(tp.PerPromptTokens) == 1 guard to avoid misapplying features to multi-prompt requests.

Bundling tokens and features into PromptTokens eliminates this assumption and the guard, and makes the per-prompt association structural rather than requiring lockstep indexing of parallel slices.

Known follow-up

flattenMMFeatures in scorer/preciseprefixcache/legacy_producer.go and convertMMFeaturesToUpstream in tokenizer/tokenizer.go are now identical (same input type, same output type, same logic). Deduplicating them is out of scope here since it crosses package boundaries and the legacy producer is deprecated.

Which issue(s) this PR fixes:

Fixes #1584

Release note (write NONE if no user-facing change):

NONE

[albertoperdomo2]

cc: @ahg-g @liu-cong

[ahg-g]

it is a bigger change, but creating a PromptTokens struct that encapsulates the tokens and the mm features is more idiomatic

type TokenizedPrompt struct {
... 
  PromptTokens []PromptToken
...
}

type  struct {
  Tokens []uint32
   MultiModalFeatures []MultiModalFeature
}

but we probably need better names, TokenizedPrompt and PromptToken are too close, perhaps ask Claude / Antigravity to suggest better names?

[albertoperdomo2]

@ahg-g anything blocking us on this one?

[bongwoobak]

burstprefix (not in this PR, pulled in by today's main-merge #1727) still uses the old TokenizedPrompt