-
Notifications
You must be signed in to change notification settings - Fork 6
Bundle namespace-credential-provisioner into management setup phase #74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
HiranAdikari
merged 1 commit into
main
from
feature/73-bundle-provisioner-into-management
Apr 28, 2026
+133
−29
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
93 changes: 93 additions & 0 deletions
93
modules/management/namespace-credential-provisioner/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Module: management/namespace-credential-provisioner | ||
|
|
||
| Deploys a long-running reconciler on the Harvester cluster that automatically provisions | ||
| credentials in every tenant namespace. This is a required part of the management phase — | ||
| deploy it after `harvester-integration` and before creating tenant workloads. | ||
|
|
||
| ## What it does | ||
|
|
||
| For every namespace labelled as a tenant namespace, the provisioner creates: | ||
|
|
||
| 1. `harvester-vm-access-<ns>` ServiceAccount with scoped RoleBindings: | ||
| - `harvesterhci.io:edit` in the tenant namespace (VM lifecycle) | ||
| - `edit` in the tenant namespace (generic Kubernetes resources) | ||
| - `harvesterhci.io:view` in `harvester-public` (read shared OS images) | ||
| 2. A long-lived SA token Secret | ||
| 3. `harvester-vm-kubeconfig` Secret in the namespace — a namespace-scoped kubeconfig | ||
| consumers use to authenticate the `harvester` Terraform provider | ||
|
|
||
| On startup the provisioner backfills any existing namespaces that are missing the | ||
| `harvester-vm-kubeconfig` Secret (upgrade path). | ||
|
|
||
| On namespace deletion it cleans up the cross-namespace `harvester-public` RoleBinding. | ||
|
|
||
| ## Why this matters | ||
|
|
||
| Without the provisioner, consumer teams cannot authenticate to Harvester to create VMs. | ||
| The alternative — handing out admin kubeconfigs or running per-team credential setup | ||
| manually — does not scale and creates security exposure. This provisioner eliminates | ||
| both problems: credentials are created automatically, scoped per namespace, and revoked | ||
| automatically when the namespace is deleted. | ||
|
|
||
| ## Deployment sequence | ||
|
|
||
| ```text | ||
| Phase 2a harvester-integration — registers Harvester with Rancher | ||
| Phase 2e namespace-credential-provisioner ← deploy here | ||
| tenant-space — creates namespaces; provisioner reacts immediately | ||
| ``` | ||
|
|
||
| The provisioner must be running before `tenant-space` creates namespaces so that | ||
| `harvester-vm-kubeconfig` is ready by the time consumer teams run `terraform apply`. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```hcl | ||
| module "provisioner" { | ||
| source = "github.com/wso2/open-cloud-datacenter//modules/management/namespace-credential-provisioner?ref=v0.8.0" | ||
|
|
||
| providers = { | ||
| kubernetes = kubernetes.harvester | ||
| } | ||
|
|
||
| harvester_api_server = "https://192.168.10.6:6443" | ||
| rancher_kubeconfig = file(var.rancher_kubeconfig_path) | ||
|
|
||
| depends_on = [module.harvester_integration] | ||
| } | ||
| ``` | ||
|
|
||
| ## Inputs | ||
|
|
||
| | Name | Description | Type | Default | Required | | ||
| |------|-------------|------|---------|----------| | ||
| | `harvester_api_server` | Harvester Kubernetes API server URL (e.g. `https://192.168.10.6:6443`) | `string` | — | yes | | ||
| | `rancher_kubeconfig` | Kubeconfig for the Rancher cluster. Used to write `harvesterconfig` secrets into `fleet-default`. | `string` | — | yes | | ||
| | `namespace` | Namespace to deploy the provisioner into | `string` | `"kube-system"` | no | | ||
| | `image` | Container image for the provisioner (needs `kubectl`, `bash`, `jq`) | `string` | `"alpine/k8s:1.32.3"` | no | | ||
|
|
||
| ## Outputs | ||
|
|
||
| | Name | Description | | ||
| |------|-------------| | ||
| | `deployment_name` | Name of the provisioner Deployment | | ||
| | `service_account_name` | ServiceAccount used by the provisioner pod | | ||
|
|
||
| ## Security | ||
|
|
||
| The provisioner SA has cluster-wide namespace watch and cross-namespace write access for | ||
| ServiceAccounts, Secrets, and RoleBindings — this is the minimum required to manage | ||
| credentials across all tenant namespaces. The credentials it creates are namespace-scoped: | ||
| each `harvester-vm-access-<ns>` SA can only act within its own namespace (plus read-only | ||
| access to `harvester-public` for shared images). | ||
|
|
||
| One project per team is strongly recommended. Within a shared project, namespace isolation | ||
| is enforced by the SA RoleBindings — not Rancher project RBAC — so consumers cannot | ||
| cross namespace boundaries even if they share a project. | ||
|
|
||
| ## Relation to `harvester-cloud-credential` | ||
|
|
||
| `workloads/harvester-cloud-credential` is deprecated. It served the same purpose | ||
| (creating per-cluster Harvester credentials) but required manual invocation per cluster. | ||
| The provisioner replaces it for all greenfield deployments. Retain the module only for | ||
| brownfield clusters that have existing credentials that cannot be migrated. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,46 +1,37 @@ | ||
| # Module: workloads/harvester-cloud-credential | ||
|
|
||
| > **This module is for infra/platform team use only.** | ||
| > **Deprecated.** Use `management/namespace-credential-provisioner` instead. | ||
| > | ||
| > If you are a consumer team provisioning your own RKE2 cluster, you do **not** need this | ||
| > module. The `namespace-credential-provisioner` (deployed in the management phase) | ||
| > automatically creates the `harvesterconfig-<cluster-name>` secret in Rancher's | ||
| > `fleet-default` namespace when it detects a new cluster in your namespace. See the | ||
| > [k8s-cluster module README](../k8s-cluster/README.md#harvester-cloud-provider-credential) | ||
| > for the consumer workflow. | ||
| > This module is retained for brownfield clusters that already have credentials created | ||
| > outside Terraform. Do not use it for new deployments — the provisioner handles this | ||
| > automatically as part of the management phase (Phase 2e). | ||
|
|
||
| Creates the `harvesterconfig-<cluster-name>` secret that the Harvester cloud provider | ||
| (CSI driver + load balancer controller) on RKE2 nodes uses to authenticate against the | ||
| Harvester Kubernetes API. Requires direct `kubernetes.harvester` and `kubernetes.rancher_local` | ||
| provider access — credentials that are only available to the platform team. | ||
| --- | ||
|
|
||
| ## When you still need this module | ||
| Creates the per-cluster Harvester cloud provider credential Secret (`harvesterconfig-<cluster-name>`) | ||
| in Rancher's `fleet-default` namespace. This secret is required by RKE2 node VMs so the | ||
| Harvester CSI driver and load balancer controller can authenticate back to Harvester. | ||
|
|
||
| Use this module in environments where the `namespace-credential-provisioner` is **not** | ||
| deployed (e.g. a standalone Harvester+Rancher setup without the management phase provisioner). | ||
| In that case, call this module once per RKE2 cluster before the cluster's first `terraform apply`. | ||
| ## When to use | ||
|
|
||
| ## Requirements | ||
| Only use this module for **brownfield clusters** that: | ||
| - Were provisioned before the `namespace-credential-provisioner` was deployed, and | ||
| - Have no existing `harvesterconfig-<cluster-name>` Secret managed by the provisioner | ||
|
|
||
| | Name | Version | | ||
| |------|---------| | ||
| | terraform | >= 1.7 | | ||
| | hashicorp/kubernetes | ~> 2.35 | | ||
|
|
||
| Requires two provider aliases: `kubernetes.harvester` (Harvester kube-apiserver) and | ||
| `kubernetes.rancher_local` (Rancher local cluster kube-apiserver). | ||
| For all other cases, deploy `management/namespace-credential-provisioner` as part of | ||
| Phase 2 and it will create the credential automatically when the cluster is detected. | ||
|
|
||
| ## Inputs | ||
|
|
||
| | Name | Description | Type | Required | | ||
| |------|-------------|------|----------| | ||
| | `cluster_name` | RKE2 cluster name (DNS-1123, max 253 chars) | `string` | yes | | ||
| | `vm_namespace` | Harvester namespace where cluster node VMs run | `string` | yes | | ||
| | `harvester_api_server` | Direct Harvester kube-apiserver URL (port 6443) | `string` | yes | | ||
| | `cluster_name` | RKE2 cluster name (used in Secret name and SA name) | `string` | yes | | ||
| | `vm_namespace` | Harvester namespace the cluster's VMs run in | `string` | yes | | ||
| | `harvester_api_server` | Harvester API server URL (e.g. `https://192.168.10.6:6443`) | `string` | yes | | ||
|
|
||
| ## Outputs | ||
|
|
||
| | Name | Description | | ||
| |------|-------------| | ||
| | `secret_name` | Secret name in `fleet-default` — pass to `k8s-cluster.cloud_provider_config_secret` | | ||
| | `service_account_name` | ServiceAccount name created in the VM namespace | | ||
| | `secret_name` | Name of the `harvesterconfig-<cluster-name>` Secret in `fleet-default` | | ||
| | `service_account_name` | Name of the ServiceAccount created in the VM namespace on Harvester | |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.