Add CEL validation rule for RemoteMCPServer with CA Cert and AllowedNamespaces (#1988)

iplay88keys · web-flow · commit 9afbe4e7bef8 · 2026-06-12T16:31:00.000-04:00
# Description

Adds a CEL admission rule on `RemoteMCPServer` that rejects setting both
`spec.allowedNamespaces` and `spec.tls.caCertSecretRef`. A
`RemoteMCPServer` that pins a CA Secret must be same-namespace-only; one
that allows cross-namespace references must not pin a CA Secret.

This closes a silent footgun in the cross-namespace + TLS combination
without changing any working behavior: the `AllowedNamespaces` field
stays, the reconciler is unchanged, and non-CA cross-namespace sharing
keeps working exactly as before.

# Why

When an agent consumes a `RemoteMCPServer` that pins a CA bundle, the
translator mounts that Secret onto the **agent's** pod by bare name
(`addTLSConfiguration`). A `SecretVolumeSource` has no namespace field —
Kubernetes always resolves it in the pod's own namespace, not the
`RemoteMCPServer`'s. So if the RMS is referenced cross-namespace, the CA
mount either:

- **dangles** — the Secret doesn't exist in the agent's namespace and
the pod fails to start; or
- **silently mounts the wrong CA** — an unrelated Secret of the same
name that happens to exist in the agent's namespace is trusted for the
upstream, with no error anywhere.

Meanwhile the controller validates and hashes the Secret in the
`RemoteMCPServer`'s namespace, so it reports `Accepted=true` while the
pod mount looks elsewhere. The cert reference itself is structurally
same-namespace (`caCertSecretRef` is a bare name with no namespace
field, resolved in the owner's namespace by design); the only thing that
broke co-location was opening the RMS object up cross-namespace via
`allowedNamespaces`. Making the two mutually exclusive removes the
unsound combination at admission time rather than letting it surface as
a runtime mount failure on the consuming agent.

# Validation rule

Spec-level `XValidation` on `RemoteMCPServerSpec`:

```
rule:    !(has(self.allowedNamespaces) &amp;&amp; has(self.allowedNamespaces.from)
            &amp;&amp; (self.allowedNamespaces.from == 'All' || self.allowedNamespaces.from == 'Selector')
            &amp;&amp; has(self.tls) &amp;&amp; has(self.tls.caCertSecretRef) &amp;&amp; size(self.tls.caCertSecretRef) &gt; 0)
message: spec.allowedNamespaces must not permit cross-namespace access (from: All or from: Selector)
         when spec.tls.caCertSecretRef is set: a pinned CA Secret is mounted onto the consuming agent's
         pod and Kubernetes resolves it in the agent's namespace, not this RemoteMCPServer's. Use
         from: Same (the default), or remove the CA Secret reference.
```

The rule only rejects the genuinely-hazardous case — a CA Secret
combined with an `allowedNamespaces` that actually permits other
namespaces (`from: All` or `from: Selector`). `from: Same` (or
omitted/empty, which defaults to same-namespace) is always allowed
alongside a CA Secret, since it carries no cross-namespace mount hazard.

Regenerated `kagent.dev_remotemcpservers.yaml` and synced the helm CRD
copy under `helm/kagent-crds/templates/`.

# Tests

Added cases to the existing envtest CEL suite (`tlsconfig_cel_test.go`),
exercised against a real kube-apiserver:

- `allowedNamespaces` `from: All` + `caCertSecretRef` → rejected.
- `allowedNamespaces` `from: Selector` + `caCertSecretRef` → rejected.
- `allowedNamespaces` `from: Same` + `caCertSecretRef` → accepted
(same-namespace, no hazard).
- `allowedNamespaces` `from: All`, no TLS → accepted (cross-namespace
sharing still works).
- `caCertSecretRef`, no `allowedNamespaces` → accepted (same-namespace
CA pinning still works).

# Compatibility

Not a breaking API change — no field is removed and no working
configuration changes behavior. The only new effect is at admission: a
manifest that combines a CA Secret with a *cross-namespace-permitting*
`allowedNamespaces` (`from: All` / `from: Selector`) is now rejected on
create/update. That combination was already non-functional (dangling or
wrong-CA mount), so this turns a silent runtime failure into an
explicit, actionable admission error. Same-namespace configurations
(`from: Same` or omitted) are unaffected, and objects already stored are
not re-validated until their next write.

---------

Signed-off-by: Jeremy Alvis &lt;jeremy.alvis@solo.io&gt;
diff --git a/go/api/config/crd/bases/kagent.dev_remotemcpservers.yaml b/go/api/config/crd/bases/kagent.dev_remotemcpservers.yaml
@@ -59,6 +59,13 @@ spec:
                   This follows the Gateway API pattern for cross-namespace route attachments.
                   If not specified, only Agents in the same namespace can reference this RemoteMCPServer.
                   See: https://gateway-api.sigs.k8s.io/guides/multiple-ns/#cross-namespace-route-attachment
+
+                  A cross-namespace-permitting value (from: All or from: Selector) is
+                  mutually exclusive with spec.tls.caCertSecretRef (enforced by a spec-level
+                  XValidation rule): a pinned CA Secret is mounted onto the consuming agent's
+                  pod by bare name and Kubernetes resolves it in the agent's namespace, not
+                  this RemoteMCPServer's, so a CA-pinning RemoteMCPServer cannot be referenced
+                  cross-namespace. from: Same (the default) is always allowed.
                 properties:
                   from:
                     default: Same
@@ -255,6 +262,15 @@ spec:
                 TLS opinion contradicts a plaintext URL. Either drop spec.tls, or
                 use https:// / a scheme-less URL.'
               rule: '!self.url.startsWith(''http://'') || !has(self.tls)'
+            - message: 'spec.allowedNamespaces must not permit cross-namespace access
+                (from: All or from: Selector) when spec.tls.caCertSecretRef is set:
+                a pinned CA Secret is mounted onto the consuming agent''s pod and
+                Kubernetes resolves it in the agent''s namespace, not this RemoteMCPServer''s.
+                Use from: Same (the default), or remove the CA Secret reference.'
+              rule: '!(has(self.allowedNamespaces) && has(self.allowedNamespaces.from)
+                && (self.allowedNamespaces.from == ''All'' || self.allowedNamespaces.from
+                == ''Selector'') && has(self.tls) && has(self.tls.caCertSecretRef)
+                && size(self.tls.caCertSecretRef) > 0)'
           status:
             description: RemoteMCPServerStatus defines the observed state of RemoteMCPServer.
             properties:
diff --git a/go/api/v1alpha2/remotemcpserver_types.go b/go/api/v1alpha2/remotemcpserver_types.go
@@ -39,6 +39,7 @@ const (
 // RemoteMCPServerSpec defines the desired state of RemoteMCPServer.
 //
 // +kubebuilder:validation:XValidation:message="spec.tls must be unset when spec.url has http:// scheme: a TLS opinion contradicts a plaintext URL. Either drop spec.tls, or use https:// / a scheme-less URL.",rule="!self.url.startsWith('http://') || !has(self.tls)"
+// +kubebuilder:validation:XValidation:message="spec.allowedNamespaces must not permit cross-namespace access (from: All or from: Selector) when spec.tls.caCertSecretRef is set: a pinned CA Secret is mounted onto the consuming agent's pod and Kubernetes resolves it in the agent's namespace, not this RemoteMCPServer's. Use from: Same (the default), or remove the CA Secret reference.",rule="!(has(self.allowedNamespaces) && has(self.allowedNamespaces.from) && (self.allowedNamespaces.from == 'All' || self.allowedNamespaces.from == 'Selector') && has(self.tls) && has(self.tls.caCertSecretRef) && size(self.tls.caCertSecretRef) > 0)"
 type RemoteMCPServerSpec struct {
 	// +required
 	Description string `json:"description"`
@@ -63,6 +64,13 @@ type RemoteMCPServerSpec struct {
 	// This follows the Gateway API pattern for cross-namespace route attachments.
 	// If not specified, only Agents in the same namespace can reference this RemoteMCPServer.
 	// See: https://gateway-api.sigs.k8s.io/guides/multiple-ns/#cross-namespace-route-attachment
+	//
+	// A cross-namespace-permitting value (from: All or from: Selector) is
+	// mutually exclusive with spec.tls.caCertSecretRef (enforced by a spec-level
+	// XValidation rule): a pinned CA Secret is mounted onto the consuming agent's
+	// pod by bare name and Kubernetes resolves it in the agent's namespace, not
+	// this RemoteMCPServer's, so a CA-pinning RemoteMCPServer cannot be referenced
+	// cross-namespace. from: Same (the default) is always allowed.
 	// +optional
 	AllowedNamespaces *AllowedNamespaces `json:"allowedNamespaces,omitempty"`
 
diff --git a/go/api/v1alpha2/tlsconfig_cel_test.go b/go/api/v1alpha2/tlsconfig_cel_test.go
@@ -217,6 +217,82 @@ func TestTLSConfigCELValidation(t *testing.T) {
 				}
 			},
 		},
+		// caCertSecretRef ⊥ cross-namespace-permitting allowedNamespaces: a pinned
+		// CA Secret can't be mounted across namespaces. from=All / from=Selector
+		// are rejected alongside a CA; from=Same (or omitted) is fine.
+		{
+			name: "RemoteMCPServer: allowedNamespaces from=All with caCertSecretRef rejected",
+			build: func() ctrl_client.Object {
+				return &RemoteMCPServer{
+					ObjectMeta: metav1.ObjectMeta{Name: "rms-allowedns-all-ca", Namespace: ns},
+					Spec: RemoteMCPServerSpec{
+						Description:       "test",
+						URL:               "https://upstream.example.com/mcp",
+						AllowedNamespaces: &AllowedNamespaces{From: NamespacesFromAll},
+						TLS:               &TLSConfig{CACertSecretRef: "ca", CACertSecretKey: "ca.crt"},
+					},
+				}
+			},
+			wantReject: "spec.allowedNamespaces must not permit cross-namespace access",
+		},
+		{
+			name: "RemoteMCPServer: allowedNamespaces from=Selector with caCertSecretRef rejected",
+			build: func() ctrl_client.Object {
+				return &RemoteMCPServer{
+					ObjectMeta: metav1.ObjectMeta{Name: "rms-allowedns-selector-ca", Namespace: ns},
+					Spec: RemoteMCPServerSpec{
+						Description: "test",
+						URL:         "https://upstream.example.com/mcp",
+						AllowedNamespaces: &AllowedNamespaces{
+							From:     NamespacesFromSelector,
+							Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"team": "x"}},
+						},
+						TLS: &TLSConfig{CACertSecretRef: "ca", CACertSecretKey: "ca.crt"},
+					},
+				}
+			},
+			wantReject: "spec.allowedNamespaces must not permit cross-namespace access",
+		},
+		{
+			name: "RemoteMCPServer: allowedNamespaces from=Same with caCertSecretRef accepted",
+			build: func() ctrl_client.Object {
+				return &RemoteMCPServer{
+					ObjectMeta: metav1.ObjectMeta{Name: "rms-allowedns-same-ca", Namespace: ns},
+					Spec: RemoteMCPServerSpec{
+						Description:       "test",
+						URL:               "https://upstream.example.com/mcp",
+						AllowedNamespaces: &AllowedNamespaces{From: NamespacesFromSame},
+						TLS:               &TLSConfig{CACertSecretRef: "ca", CACertSecretKey: "ca.crt"},
+					},
+				}
+			},
+		},
+		{
+			name: "RemoteMCPServer: allowedNamespaces from=All without CA accepted",
+			build: func() ctrl_client.Object {
+				return &RemoteMCPServer{
+					ObjectMeta: metav1.ObjectMeta{Name: "rms-allowedns-no-ca", Namespace: ns},
+					Spec: RemoteMCPServerSpec{
+						Description:       "test",
+						URL:               "https://upstream.example.com/mcp",
+						AllowedNamespaces: &AllowedNamespaces{From: NamespacesFromAll},
+					},
+				}
+			},
+		},
+		{
+			name: "RemoteMCPServer: caCertSecretRef without allowedNamespaces accepted",
+			build: func() ctrl_client.Object {
+				return &RemoteMCPServer{
+					ObjectMeta: metav1.ObjectMeta{Name: "rms-ca-no-allowedns", Namespace: ns},
+					Spec: RemoteMCPServerSpec{
+						Description: "test",
+						URL:         "https://upstream.example.com/mcp",
+						TLS:         &TLSConfig{CACertSecretRef: "ca", CACertSecretKey: "ca.crt"},
+					},
+				}
+			},
+		},
 	}
 
 	for _, c := range cases {
diff --git a/helm/kagent-crds/templates/kagent.dev_remotemcpservers.yaml b/helm/kagent-crds/templates/kagent.dev_remotemcpservers.yaml
@@ -59,6 +59,13 @@ spec:
                   This follows the Gateway API pattern for cross-namespace route attachments.
                   If not specified, only Agents in the same namespace can reference this RemoteMCPServer.
                   See: https://gateway-api.sigs.k8s.io/guides/multiple-ns/#cross-namespace-route-attachment
+
+                  A cross-namespace-permitting value (from: All or from: Selector) is
+                  mutually exclusive with spec.tls.caCertSecretRef (enforced by a spec-level
+                  XValidation rule): a pinned CA Secret is mounted onto the consuming agent's
+                  pod by bare name and Kubernetes resolves it in the agent's namespace, not
+                  this RemoteMCPServer's, so a CA-pinning RemoteMCPServer cannot be referenced
+                  cross-namespace. from: Same (the default) is always allowed.
                 properties:
                   from:
                     default: Same
@@ -255,6 +262,15 @@ spec:
                 TLS opinion contradicts a plaintext URL. Either drop spec.tls, or
                 use https:// / a scheme-less URL.'
               rule: '!self.url.startsWith(''http://'') || !has(self.tls)'
+            - message: 'spec.allowedNamespaces must not permit cross-namespace access
+                (from: All or from: Selector) when spec.tls.caCertSecretRef is set:
+                a pinned CA Secret is mounted onto the consuming agent''s pod and
+                Kubernetes resolves it in the agent''s namespace, not this RemoteMCPServer''s.
+                Use from: Same (the default), or remove the CA Secret reference.'
+              rule: '!(has(self.allowedNamespaces) && has(self.allowedNamespaces.from)
+                && (self.allowedNamespaces.from == ''All'' || self.allowedNamespaces.from
+                == ''Selector'') && has(self.tls) && has(self.tls.caCertSecretRef)
+                && size(self.tls.caCertSecretRef) > 0)'
           status:
             description: RemoteMCPServerStatus defines the observed state of RemoteMCPServer.
             properties:
