docs(enterprise): add Kubernetes Operator Bring-Your-Own-Cluster (BYOC) guide

[peasee]

🗣 Description

Documents the operator's managed mode: registration in Spice.ai Cloud, enrollment-token install, identity lifecycle, kubelet metrics TLS options, pause semantics, and troubleshooting for common failure modes. Adds the spice.managedMode Helm values and --managed-mode-* CLI flags to the operator Overview, the managed-mode scrape/export counters to Operator Metrics, and cross-links from the User Guide.

🔨 Related Issues

🤔 Concerns

[Copilot]

Pull request overview

Adds comprehensive BYOC managed-mode documentation for connecting customer Kubernetes clusters to Spice.ai Cloud.

Changes:

• Adds setup, identity, security, pause, metrics, and troubleshooting guidance.
• Documents managed-mode Helm values and CLI flags.
• Adds navigation and cross-references across operator documentation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
enterprise/SUMMARY.md	Adds BYOC navigation.
enterprise/kubernetes/user-guide.md	Links users to BYOC setup.
enterprise/kubernetes/README.md	Documents managed-mode configuration.
enterprise/kubernetes/metrics.md	Documents managed-mode metrics.
enterprise/kubernetes/byoc.md	Adds the complete BYOC guide.

Suppressed comments (2)

enterprise/kubernetes/byoc.md:256

• This pre-provisioned path explicitly skips enrollment, so serverCaSecret cannot make enrollment trust the private CA here. Referring to enrollment makes the trust behavior ambiguous; it should identify the control-stream connection instead.

With a pre-provisioned identity the operator does not enroll and does not run its renewal loop — your certificate issuer owns rotation. `enrollmentToken`, `enrollmentTokenSecret`, and `enrollUrl` are ignored on this path. For a control plane that serves a certificate from a private CA, additionally set `serverCaSecret` so enrollment and reconnects trust it; concatenate multiple CAs into one PEM bundle if needed.

enterprise/kubernetes/byoc.md:261

• This restriction contradicts the re-enrollment workflow below and the 409 troubleshooting entry, both of which tell an already-enrolled cluster to obtain a fresh token. These instructions cannot all be followed as written. Clarify whether an existing registration can issue a re-enrollment token; if it cannot, those recovery paths must instead require decommissioning and registering a new cluster.

- **Rotate the enrollment token** — from the cluster's **Setup** tab (before first enrollment) or **Admin** tab. Issuing a new token revokes the outstanding one. Tokens can only be rotated before the cluster enrolls.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (5)

enterprise/kubernetes/byoc.md:262

• The statement that tokens can only be rotated before enrollment conflicts with the next bullet's procedure for re-enrolling an existing registration and with the 409 troubleshooting row, which tells users to issue a new token after a previous install enrolled. Clarify whether the Admin tab can issue a post-enrollment token; if it cannot, the re-enrollment guidance must instead require a new registration.

- **Rotate the enrollment token** — from the cluster's **Setup** tab (before first enrollment) or **Admin** tab. Issuing a new token revokes the outstanding one. Tokens can only be rotated before the cluster enrolls.

enterprise/kubernetes/byoc.md:173

• An Approved CSR only records authorization; it does not prove that a certificate was issued or that its chain is trusted by the operator. The assertion that the default configuration works can therefore misdirect users whose approved request is still unissued or uses another CA. Check the issued certificate/served chain or rely on the tls_error counter.

Approved requests mean the cluster issues kubelet serving certificates from its own CA, and the default configuration works. **No output is inconclusive** — Kubernetes garbage-collects CSR objects, and kubelet certificates can be provisioned without a retained CSR — so do not switch to `kubeletInsecureTls` on that signal alone. Rely on the `tls_error` count above, or inspect the issuer of the certificate a kubelet serves on port `10250`.

enterprise/kubernetes/metrics.md:134

• This description says batches were sent, but the documented results include queue_full, empty, and resource_too_large, which explicitly represent outcomes where no batch was sent. Describe the counter as export attempts/outcomes rather than successful sends.

| `spiceai_operator_managed_metrics_export_total`                  | `result`              | Export batches sent to Spice.ai Cloud.                                  |

enterprise/kubernetes/byoc.md:257

• This paragraph first says the pre-provisioned path skips enrollment and ignores enrollUrl, then says serverCaSecret is used to make enrollment trust the CA. On this path it applies to the control-stream connection, not enrollment.

This issue also appears on line 262 of the same file.

With a pre-provisioned identity the operator does not enroll and does not run its renewal loop — your certificate issuer owns rotation. `enrollmentToken`, `enrollmentTokenSecret`, and `enrollUrl` are ignored on this path. For a control plane that serves a certificate from a private CA, additionally set `serverCaSecret` so enrollment and reconnects trust it; concatenate multiple CAs into one PEM bundle if needed.

enterprise/kubernetes/byoc.md:31

• The Kubernetes cluster does not pull the OCI chart; the Helm client does. As written, firewall administrators may permit registry access from cluster nodes while the workstation or CI runner performing the install still cannot fetch the chart. Distinguish Helm-client chart access, node image pulls, and operator control-plane egress.

The cluster also pulls the operator chart and the Spice runtime images from the AWS Marketplace registry, `709825985650.dkr.ecr.us-east-1.amazonaws.com`. Allow egress to all of these hosts on port 443.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (6)

enterprise/kubernetes/byoc.md:31

• This attributes the chart download to the cluster, but Helm pulls the chart from the machine where the command runs; only the container images are pulled by cluster nodes. As written, the egress checklist can leave the Helm client without registry access while requiring unnecessary chart-pull access from nodes.

The cluster also pulls the operator chart and the Spice runtime images from the AWS Marketplace registry, `709825985650.dkr.ecr.us-east-1.amazonaws.com`. Allow egress to all of these hosts on port 443.

enterprise/kubernetes/byoc.md:186

• The Helm default conflicts with the Overview: README.md:74 documents an empty enrollUrl Helm value that selects the Spice.ai Cloud URL, while this row says the Helm value itself defaults to the URL. Align both tables with the actual chart default so values-file users know whether the flag is rendered or the binary default is used.

| `enrollUrl`                 | `https://api.spice.ai/v1/cloud-connect` | Base URL of the enrollment API. Leave empty for Spice.ai Cloud; set only for a self-hosted control plane.                    |

enterprise/kubernetes/byoc.md:173

• An approved kubelet-serving CSR does not guarantee that the certificate currently served on port 10250 chains to the CA used by the operator or contains a SAN for the node address selected by the scraper. Treating approval as proof that the default works can send users away from the actual TLS failure; the live certificate or tls_error result remains authoritative.

Approved requests mean the cluster issues kubelet serving certificates from its own CA, and the default configuration works. **No output is inconclusive** — Kubernetes garbage-collects CSR objects, and kubelet certificates can be provisioned without a retained CSR — so do not switch to `kubeletInsecureTls` on that signal alone. Rely on the `tls_error` count above, or inspect the issuer of the certificate a kubelet serves on port `10250`.

enterprise/kubernetes/byoc.md:257

• This paragraph has just established that the pre-provisioned path never enrolls and ignores enrollUrl, so saying that serverCaSecret makes “enrollment” trust the CA is contradictory. In this path the trust anchor applies to the control connection and reconnects.

With a pre-provisioned identity the operator does not enroll and does not run its renewal loop — your certificate issuer owns rotation. `enrollmentToken`, `enrollmentTokenSecret`, and `enrollUrl` are ignored on this path. For a control plane that serves a certificate from a private CA, additionally set `serverCaSecret` so enrollment and reconnects trust it; concatenate multiple CAs into one PEM bundle if needed.

enterprise/kubernetes/byoc.md:262

• This restriction conflicts with every documented recovery path for an already-enrolled cluster: lines 265, 303, 313, and 323 all require obtaining a fresh enrollment token after deleting or losing an enrolled identity. Clarify the portal/API action that permits post-enrollment re-enrollment, or change those recovery paths to decommission and register a new cluster; as written, users cannot perform the stated recovery procedure.

- **Rotate the enrollment token** — from the cluster's **Setup** tab (before first enrollment) or **Admin** tab. Issuing a new token revokes the outstanding one. Tokens can only be rotated before the cluster enrolls.

enterprise/kubernetes/metrics.md:134

• The description says these batches were sent, but the documented results include enqueued, queue_full, empty, and resource_too_large; the latter three do not represent successful sends. Describe this as export processing/queue outcomes so dashboards do not interpret every increment as a cloud delivery.

| `spiceai_operator_managed_metrics_export_total`                  | `result`              | Export batches sent to Spice.ai Cloud.                                  |