Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,20 @@ This phase uses four modules, typically applied together:

**Provider dependencies**: `rancher/rancher2 ~> 3.0`

#### 2e. namespace-credential-provisioner (`modules/management/namespace-credential-provisioner`)

- Deploys a long-running reconciler on the Harvester cluster that watches for tenant namespaces.
- For each namespace, automatically creates a scoped ServiceAccount, RoleBindings, and a
`harvester-vm-kubeconfig` Secret that consumer teams use to authenticate the `harvester`
Terraform provider — no admin involvement, no file handover.
- Backfills existing namespaces on startup (safe to deploy to running clusters).
- Cleans up cross-namespace RoleBindings when a namespace is deleted.

**Must be deployed before `tenant-space` creates namespaces** so that credentials are
ready by the time consumer teams run `terraform apply`.

**Provider dependencies**: `hashicorp/kubernetes >= 2.0`

---

### Phase 3 — Identity & Monitoring
Expand Down Expand Up @@ -203,7 +217,13 @@ This phase configures the `asgardeo` provider (or equivalent OIDC configuration
│ projects/namespaces ready
┌───────────────────┐
┌─────────────────────────┐
│ namespace-credential- │
│ provisioner (Phase 2e) │
└──────────┬──────────────┘
│ harvesterconfig + harvester-vm-kubeconfig per namespace
┌───────────────────┐
│ identity │
│ (Phase 3a) │
└────────┬──────────┘
Expand Down
93 changes: 93 additions & 0 deletions modules/management/namespace-credential-provisioner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Module: management/namespace-credential-provisioner

Deploys a long-running reconciler on the Harvester cluster that automatically provisions
credentials in every tenant namespace. This is a required part of the management phase —
deploy it after `harvester-integration` and before creating tenant workloads.

## What it does

For every namespace labelled as a tenant namespace, the provisioner creates:

1. `harvester-vm-access-<ns>` ServiceAccount with scoped RoleBindings:
- `harvesterhci.io:edit` in the tenant namespace (VM lifecycle)
- `edit` in the tenant namespace (generic Kubernetes resources)
- `harvesterhci.io:view` in `harvester-public` (read shared OS images)
2. A long-lived SA token Secret
3. `harvester-vm-kubeconfig` Secret in the namespace — a namespace-scoped kubeconfig
consumers use to authenticate the `harvester` Terraform provider

On startup the provisioner backfills any existing namespaces that are missing the
`harvester-vm-kubeconfig` Secret (upgrade path).

On namespace deletion it cleans up the cross-namespace `harvester-public` RoleBinding.

## Why this matters

Without the provisioner, consumer teams cannot authenticate to Harvester to create VMs.
The alternative — handing out admin kubeconfigs or running per-team credential setup
manually — does not scale and creates security exposure. This provisioner eliminates
both problems: credentials are created automatically, scoped per namespace, and revoked
automatically when the namespace is deleted.

## Deployment sequence

```text
Phase 2a harvester-integration — registers Harvester with Rancher
Phase 2e namespace-credential-provisioner ← deploy here
tenant-space — creates namespaces; provisioner reacts immediately
```

The provisioner must be running before `tenant-space` creates namespaces so that
`harvester-vm-kubeconfig` is ready by the time consumer teams run `terraform apply`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Usage

```hcl
module "provisioner" {
source = "github.com/wso2/open-cloud-datacenter//modules/management/namespace-credential-provisioner?ref=v0.8.0"

providers = {
kubernetes = kubernetes.harvester
}

harvester_api_server = "https://192.168.10.6:6443"
rancher_kubeconfig = file(var.rancher_kubeconfig_path)

depends_on = [module.harvester_integration]
}
```

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|----------|
| `harvester_api_server` | Harvester Kubernetes API server URL (e.g. `https://192.168.10.6:6443`) | `string` | — | yes |
| `rancher_kubeconfig` | Kubeconfig for the Rancher cluster. Used to write `harvesterconfig` secrets into `fleet-default`. | `string` | — | yes |
| `namespace` | Namespace to deploy the provisioner into | `string` | `"kube-system"` | no |
| `image` | Container image for the provisioner (needs `kubectl`, `bash`, `jq`) | `string` | `"alpine/k8s:1.32.3"` | no |

## Outputs

| Name | Description |
|------|-------------|
| `deployment_name` | Name of the provisioner Deployment |
| `service_account_name` | ServiceAccount used by the provisioner pod |

## Security

The provisioner SA has cluster-wide namespace watch and cross-namespace write access for
ServiceAccounts, Secrets, and RoleBindings — this is the minimum required to manage
credentials across all tenant namespaces. The credentials it creates are namespace-scoped:
each `harvester-vm-access-<ns>` SA can only act within its own namespace (plus read-only
access to `harvester-public` for shared images).

One project per team is strongly recommended. Within a shared project, namespace isolation
is enforced by the SA RoleBindings — not Rancher project RBAC — so consumers cannot
cross namespace boundaries even if they share a project.

## Relation to `harvester-cloud-credential`

`workloads/harvester-cloud-credential` is deprecated. It served the same purpose
(creating per-cluster Harvester credentials) but required manual invocation per cluster.
The provisioner replaces it for all greenfield deployments. Retain the module only for
brownfield clusters that have existing credentials that cannot be migrated.
47 changes: 19 additions & 28 deletions modules/workloads/harvester-cloud-credential/README.md
Original file line number Diff line number Diff line change
@@ -1,46 +1,37 @@
# Module: workloads/harvester-cloud-credential

> **This module is for infra/platform team use only.**
> **Deprecated.** Use `management/namespace-credential-provisioner` instead.
>
> If you are a consumer team provisioning your own RKE2 cluster, you do **not** need this
> module. The `namespace-credential-provisioner` (deployed in the management phase)
> automatically creates the `harvesterconfig-<cluster-name>` secret in Rancher's
> `fleet-default` namespace when it detects a new cluster in your namespace. See the
> [k8s-cluster module README](../k8s-cluster/README.md#harvester-cloud-provider-credential)
> for the consumer workflow.
> This module is retained for brownfield clusters that already have credentials created
> outside Terraform. Do not use it for new deployments — the provisioner handles this
> automatically as part of the management phase (Phase 2e).

Creates the `harvesterconfig-<cluster-name>` secret that the Harvester cloud provider
(CSI driver + load balancer controller) on RKE2 nodes uses to authenticate against the
Harvester Kubernetes API. Requires direct `kubernetes.harvester` and `kubernetes.rancher_local`
provider access — credentials that are only available to the platform team.
---

## When you still need this module
Creates the per-cluster Harvester cloud provider credential Secret (`harvesterconfig-<cluster-name>`)
in Rancher's `fleet-default` namespace. This secret is required by RKE2 node VMs so the
Harvester CSI driver and load balancer controller can authenticate back to Harvester.

Use this module in environments where the `namespace-credential-provisioner` is **not**
deployed (e.g. a standalone Harvester+Rancher setup without the management phase provisioner).
In that case, call this module once per RKE2 cluster before the cluster's first `terraform apply`.
## When to use

## Requirements
Only use this module for **brownfield clusters** that:
- Were provisioned before the `namespace-credential-provisioner` was deployed, and
- Have no existing `harvesterconfig-<cluster-name>` Secret managed by the provisioner

| Name | Version |
|------|---------|
| terraform | >= 1.7 |
| hashicorp/kubernetes | ~> 2.35 |

Requires two provider aliases: `kubernetes.harvester` (Harvester kube-apiserver) and
`kubernetes.rancher_local` (Rancher local cluster kube-apiserver).
For all other cases, deploy `management/namespace-credential-provisioner` as part of
Phase 2 and it will create the credential automatically when the cluster is detected.

## Inputs

| Name | Description | Type | Required |
|------|-------------|------|----------|
| `cluster_name` | RKE2 cluster name (DNS-1123, max 253 chars) | `string` | yes |
| `vm_namespace` | Harvester namespace where cluster node VMs run | `string` | yes |
| `harvester_api_server` | Direct Harvester kube-apiserver URL (port 6443) | `string` | yes |
| `cluster_name` | RKE2 cluster name (used in Secret name and SA name) | `string` | yes |
| `vm_namespace` | Harvester namespace the cluster's VMs run in | `string` | yes |
| `harvester_api_server` | Harvester API server URL (e.g. `https://192.168.10.6:6443`) | `string` | yes |

## Outputs

| Name | Description |
|------|-------------|
| `secret_name` | Secret name in `fleet-default` — pass to `k8s-cluster.cloud_provider_config_secret` |
| `service_account_name` | ServiceAccount name created in the VM namespace |
| `secret_name` | Name of the `harvesterconfig-<cluster-name>` Secret in `fleet-default` |
| `service_account_name` | Name of the ServiceAccount created in the VM namespace on Harvester |