You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(enterprise): clarify BYOC secret encryption boundary and kubelet TLS diagnosis
The HPKE sealing protects secret values in transit only; at-rest protection
follows the cluster's Secret encryption and access controls. The
kubelet-serving CSR check is inconclusive when CSR objects have been
garbage-collected, so lead with the operator's tls_error signal instead.
Copy file name to clipboardExpand all lines: enterprise/kubernetes/byoc.md
+10-9Lines changed: 10 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -149,14 +149,9 @@ The kubelet scrape does not use the operator's own credentials. The chart create
149
149
150
150
### Kubelet TLS verification
151
151
152
-
The kubelet serves `/metrics/resource` over HTTPS. By default, the operator **verifies** the kubelet's serving certificate against the cluster CA and refuses to scrape a kubelet it cannot verify. Many clusters — including default **EKS, GKE, and AKS** clusters — use self-signed kubelet serving certificates that fail this check. To find out which case applies to your cluster, run:
152
+
The kubelet serves `/metrics/resource` over HTTPS. By default, the operator **verifies** the kubelet's serving certificate against the cluster CA and refuses to scrape a kubelet it cannot verify. Many clusters — including default **EKS, GKE, and AKS** clusters — use self-signed kubelet serving certificates that fail this check.
153
153
154
-
```bash
155
-
kubectl get csr | grep kubelet-serving
156
-
```
157
-
158
-
-**Approved requests appear** — your cluster issues kubelet certificates from its own CA. The default configuration works; set neither option below.
159
-
-**No output** — your kubelets self-sign. Container CPU/memory collection needs one of these two options (set at most one):
154
+
Start with the default (verification enabled) and let the operator tell you whether a change is needed. When the kubelet certificates cannot be verified, the operator logs the reason at startup and counts each skipped scrape on the metric `spiceai_operator_managed_metrics_scrape_total{source="kubelet",result="tls_error"}`. Spice runtime metrics keep flowing regardless — only the container CPU/memory graphs stay empty. A rising `tls_error` count means container CPU/memory collection needs one of these two options (set at most one):
@@ -169,7 +164,13 @@ These options mirror `metrics-server`'s `--kubelet-certificate-authority` and `-
169
164
With `kubeletInsecureTls: true`, the scrape presents its metrics-only token to whatever answers on the node's address, without proof it is the kubelet. That token can only read node metrics — nothing else — but prefer `kubeletCaSecret` wherever the cluster can provide a CA. The operator logs a warning at startup while verification is disabled.
170
165
{% endhint %}
171
166
172
-
If neither option is set and the kubelet certificates cannot be verified, the operator logs the reason at startup and counts each skipped scrape on the metric `spiceai_operator_managed_metrics_scrape_total{source="kubelet",result="tls_error"}`. Spice runtime metrics keep flowing regardless — only the container CPU/memory graphs stay empty.
167
+
To confirm how your cluster issues kubelet serving certificates, check for approved `kubelet-serving` certificate signing requests:
168
+
169
+
```bash
170
+
kubectl get csr | grep kubelet-serving
171
+
```
172
+
173
+
Approved requests mean the cluster issues kubelet serving certificates from its own CA, and the default configuration works. **No output is inconclusive** — Kubernetes garbage-collects CSR objects, and kubelet certificates can be provisioned without a retained CSR — so do not switch to `kubeletInsecureTls` on that signal alone. Rely on the `tls_error` count above, or inspect the issuer of the certificate a kubelet serves on port `10250`.
173
174
174
175
## Configuration reference
175
176
@@ -227,7 +228,7 @@ Enabling managed mode adds these objects, beyond the standard operator installat
227
228
-**ConfigMap** — `spice-managed-mode-pause` in the operator's namespace, used as a pause gate (see below).
228
229
-**Namespaces** — the operator creates the application namespace before it applies a manifest sent from Spice.ai Cloud, when that namespace does not exist yet.
229
230
230
-
Secret values sent from Spice.ai Cloud are sealed with two layers of public-key encryption (HPKE): one to a key that persists across restarts, and one to a key that exists only in the operator's memory for the current connection. There is no plaintext path.
231
+
Secret values sent from Spice.ai Cloud are sealed with two layers of public-key encryption (HPKE): one to a key that persists across restarts, and one to a key that exists only in the operator's memory for the current connection. There is no plaintext delivery path — the operator rejects a payload that fails either layer. This encryption protects the values in transit only: after decryption, the operator stores them as standard Kubernetes Secrets, so protection at rest depends on your cluster's Secret encryption and access controls.
0 commit comments