Keep replacing route policies out of shared clusters

[zhaohuabing]

Keep replacing route policies out of shared clusters

When two rules reference the same backend, but one has a more-specific BackendTrafficPolicy with mergeType unset, that policy must completely replace the gateway policy. Because merge eligibility only treats fields explicitly set by the route policy as cluster-scoped, both rules can still share a BackendCluster; this unconditional loop then installs the gateway policy's circuit breaker, health check, load balancer, and other cluster settings on that shared cluster, so the replacing rule still inherits settings it explicitly overrides. Such routes must be demerged or clusters must be partitioned by their effective policy set.

Useful? React with 👍 / 👎.

Originally posted by @chatgpt-codex-connector[bot] in #9477 (comment)

[zhaohuabing]

cc @muwaqar-cflt

[muwaqar-cflt]

@zhaohuabing this seems a duplicate of #9541, no?

[zhaohuabing]

I think the symptoms are different: #9541 is about RoutingType, this one is about traffic policy override.

Example:

# Both routes point at the same Service, so they dedup to one BackendCluster.
kind: HTTPRoute
metadata: {name: route-a, namespace: default}
spec:
  parentRefs: [{name: eg}]
  rules: [{backendRefs: [{name: foo, port: 8080}]}]
---
kind: HTTPRoute
metadata: {name: route-b, namespace: default}
spec:
  parentRefs: [{name: eg}]
  rules: [{backendRefs: [{name: foo, port: 8080}]}]
---
kind: BackendTrafficPolicy
metadata: {name: btp-gw}
spec:
  targetRefs: [{group: gateway.networking.k8s.io, kind: Gateway, name: eg}]
  circuitBreaker: {maxConnections: 10}
  healthCheck: {active: {type: HTTP, http: {path: /healthz}, interval: 3s}}
  loadBalancer: {type: LeastRequest}
---
kind: BackendTrafficPolicy
metadata: {name: btp-a}
spec:
  targetRefs: [{group: gateway.networking.k8s.io, kind: HTTPRoute, name: route-a}]
  faultInjection: {abort: {httpStatus: 503, percentage: 10}}
  # mergeType intentionally omitted → btp-a fully replaces btp-gw for route-a

Expected for route-a: fault injection only. No circuit breaker, no active health check, default round-robin — the gateway policy is replaced wholesale.

What actually happens:

┌─────────────────────────────────────────────────┬─────────────────────────────────────────────────────┐
│                      step                       │                       result                        │
├─────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ btpSpecHasClusterScopedFields(btp-a.Spec)       │ false — faultInjection isn't in the list            │
├─────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ hasRouteLevelClusterSettings for route-a's rule │ false → merge-eligible                              │
├─────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ getOrCreateBackendCluster for foo:8080          │ route-a and route-b get the same *ir.BackendCluster │
├─────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ route-level pass                                │ r.Traffic on route-a's IR route = faultInjection ✓  │
├─────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ gateway pass, r.Traffic != nil guard            │ route-a's route skipped ✓                           │
├─────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ gateway pass, line 1498 loop                    │ bc.Traffic = btp-gw's tf on the shared cluster ✗    │
└─────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘

CDS output: route-a's traffic lands on a cluster with maxConnections: 10, active HTTP health checking, and LeastRequest — every setting it opted out of.

[muwaqar-cflt]

right, the symptom is different, #9541 is about RoutingType index, this is about ClusterSettings index, but the proposed fix muwaqar#4 covers both scenarios. I am just waiting on #9590 to land first independently so that it can be cherry-picked to older releases as per need, and then follow-up with fix for this issue. Happy to keep this tracking issue open till then.