Problem
Private AI Gateway does not provide consistent caller control over prefix-cache partitioning.
For the active Kimi-K2.6 providers:
- Chutes forwards a caller's optional
cache_salt to vLLM.
- Tinfoil strips
cache_salt and derives its own value. It accepts the router-only user_cache_secret as the caller-controlled input, but the gateway does not currently translate the public field.
As a result, the same cache_salt request has different behavior across providers.
Desired behavior
Keep cache_salt caller-controlled across providers:
- Forward
cache_salt unchanged to Chutes and direct vLLM providers.
- Translate
cache_salt to user_cache_secret for Tinfoil, which derives the final salt.
- Do not derive, overwrite, or bind the partition to authenticated Redpill tenant identity.
- Preserve current provider behavior when the caller omits
cache_salt.
Callers may choose distinct values for isolation or reuse a value to share a cache namespace. This is user-controlled partitioning, not a gateway-enforced tenant boundary.
Acceptance criteria
- A supplied
cache_salt reaches Chutes and direct vLLM providers unchanged.
- The same public field reaches Tinfoil as
user_cache_secret.
- Omitting
cache_salt does not add a gateway-generated partition.
- Tests cover provider-specific request transformation and receipt consistency.
- Documentation explains the caller-controlled semantics and provider translation.
Evidence
Tinfoil's attested router strips cache_salt, consumes user_cache_secret, and derives the final salt: https://github.com/tinfoilsh/confidential-model-router/blob/v0.0.118/cache_salt.go
Problem
Private AI Gateway does not provide consistent caller control over prefix-cache partitioning.
For the active Kimi-K2.6 providers:
cache_saltto vLLM.cache_saltand derives its own value. It accepts the router-onlyuser_cache_secretas the caller-controlled input, but the gateway does not currently translate the public field.As a result, the same
cache_saltrequest has different behavior across providers.Desired behavior
Keep
cache_saltcaller-controlled across providers:cache_saltunchanged to Chutes and direct vLLM providers.cache_salttouser_cache_secretfor Tinfoil, which derives the final salt.cache_salt.Callers may choose distinct values for isolation or reuse a value to share a cache namespace. This is user-controlled partitioning, not a gateway-enforced tenant boundary.
Acceptance criteria
cache_saltreaches Chutes and direct vLLM providers unchanged.user_cache_secret.cache_saltdoes not add a gateway-generated partition.Evidence
Tinfoil's attested router strips
cache_salt, consumesuser_cache_secret, and derives the final salt: https://github.com/tinfoilsh/confidential-model-router/blob/v0.0.118/cache_salt.go