[Issue 42 U3.1.4] VLM preprocessing-policy cache identity

[Zigfreidish]

Parent

#1432

Governing plan: docs/plans/2026-04-26-issue-42-multimodal-fast-paths.md

What to build

Add a request-scoped VLM preprocessing-policy contract so operator pixel controls are routed only to supported image processors and become part of the image-feature cache identity.

This slice comes from the latest #42 watch-log note about min_pixels, max_pixels, resized_height, and resized_width. It should make repeated-image reuse safe when the same bytes are decoded under different preprocessing policies.

Acceptance criteria

• Supported pixel-policy fields (min_pixels, max_pixels, resized_height, resized_width) are normalized into a stable preprocessing-policy receipt before image processing.
• Allowlisted pixel controls are forwarded only to the image processor; tokenizer kwargs never receive image pixel-policy fields.
• Unsupported preprocessing fields fail closed or are omitted with a typed unsupported reason before decode.
• Image-feature cache keys and evidence receipts include the normalized preprocessing policy so identical bytes with different policy values do not collide.
• Repeated-image fixtures distinguish cache hits for identical bytes with the same policy from misses when policy changes.

Blocked by

• [Issue 42 U3.1.1] Real image feature cache #1456

[Akane-CN]

Watch-log follow-up — preprocessing identity must include processor-declared tensor layout.

• Design principle: image-feature cache keys are unsafe if they only include bytes and resize controls. Processor-declared layout/format fields also affect pixel tensors, so they need to be normalized into the preprocessing-policy receipt before cache lookup.
• Melix-specific goal: include processor input-data format and layout normalization alongside min/max/resized pixel controls in VLM preprocessing receipts and cache identity.
• Measurable target: identical image bytes processed under different declared layouts cannot collide; cache evidence records layout, resolved resize policy, and supported/unsupported policy fields.
• Next executable slice: extend the U3.1.4 fixture matrix with channels-first/channels-last processor configs and assert separate cache keys plus typed unsupported-field handling.

[Akane-CN]

Watch-log follow-up: preserve model-specific VLM prompt kwargs before cache identity

• Design principle: VLM preprocessing policy is not just pixel size; model-specific language prompt kwargs and position IDs can change feature layout and therefore must be routed deliberately and included in receipts.
• Melix-specific goal: avoid treating token_type_ids, multimodal token-type IDs, image/video position IDs, soft-token counts, frame rate, or resized media policy as interchangeable tokenizer extras. They should be allowlisted per processor/model family and become part of feature-cache identity when they affect generated embeddings.
• Measurable target: fixtures where two requests share identical media bytes but differ in position IDs, fps/frame sampling, soft-token counts, or processor sizing must either produce distinct cache keys or a typed unsupported-policy refusal; target zero tokenizer-kwarg leakage of media-only controls.
• Next executable slice: add a VLM ProcessorPolicyReceipt emitted before preprocessing, then update [Issue 42 U3.1.4] VLM preprocessing-policy cache identity #1552 cache-key tests to cover model-specific kwargs in addition to the existing pixel-policy fields.

[Akane-CN]

Watch-log update: processor config keys must be capability-checked instead of silently dropping media inputs.

• Design principle: VLM processor construction should accept known harmless config keys, reject unsupported ones with typed reasons, and never fall back to text-only tokenization without an operator-visible receipt.
• Melix-specific goal: keep image/video requests fail-closed when processor kwargs drift, and include the normalized preprocessing policy in cache identity so identical media bytes are not reused under a different decode policy.
• Measurable target: fixture with image/video processor config keys such as color conversion, sampling, resize, and metadata fields proves media is still routed; unsupported fields produce {unsupported_preprocessing_field} instead of silent text-only execution.
• Next executable slice: add a processor-kwargs admission layer before feature extraction and include {accepted_fields, ignored_fields, unsupported_fields, media_inputs_preserved} in VLM evidence.

[Akane-CN]

Design principle: multimodal fast paths must be fidelity-gated by the effective preprocessing/decode policy, not just by model family or image bytes.

Melix-specific goal: make VLM image-feature cache identity and evidence receipts include fidelity-affecting choices such as reference-image dtype conversion, tiled VAE/decode mode, sequence bucketing, and explicit pixel policy. Unsafe defaults should fail closed or fall back to the slower path when a canary detects visible/color drift.

Measurable target: add a fixture where identical image bytes with different preprocessing/decode policy produce distinct cache keys, and a smoke canary that rejects a fast-path promotion when a tiled/decode policy changes pixel statistics beyond a small threshold. Receipt should report policy_cache_key, decode_mode, dtype_policy, and fallback_reason when disabled.

Next executable slice: extend the current preprocessing-policy normalizer to cover decode/fidelity policy fields, then add one repeated-image cache test and one golden-image drift canary before enabling any tiled or bucketed shortcut by default.