Problem. There's no way to split traffic across multiple llm Instances (e.g. two versions of a model) behind one endpoint for canary, blue-green, or A/B rollouts.
Impact. Safe model upgrades require external routing; today an upgrade is effectively in-place.
Current workaround. None.
Upstream. Router.Route Group + Weight (Gateway API backendRef weight semantics; members sharing a group split traffic).
Scope. Expose a routing group + weight so several Instances can front one endpoint with weighted traffic.
Problem. There's no way to split traffic across multiple
llmInstances (e.g. two versions of a model) behind one endpoint for canary, blue-green, or A/B rollouts.Impact. Safe model upgrades require external routing; today an upgrade is effectively in-place.
Current workaround. None.
Upstream.
Router.RouteGroup+Weight(Gateway API backendRef weight semantics; members sharing a group split traffic).Scope. Expose a routing group + weight so several Instances can front one endpoint with weighted traffic.