Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 22 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,15 @@ ArgoCD**, so the whole platform can be torn down and rebuilt from this repo.
```

The cluster is **AKS** in **Sweden Central**: a single `Standard_D4s_v6` node
(4 vCPU / 16 GB, manual scaling), Kubernetes 1.36.2, **Cilium** eBPF dataplane
enforcing NetworkPolicy, Workload Identity + OIDC, and the Key Vault CSI add-on.
See [`infra/aks.bicep`](infra/aks.bicep).
(4 vCPU / 16 GB, manual scaling), Kubernetes **1.36** (the minor only — the patch
channel owns the patch level), **Cilium** eBPF dataplane enforcing NetworkPolicy,
Workload Identity + OIDC, and the Key Vault CSI add-on. See
[`infra/aks.bicep`](infra/aks.bicep).

What the cluster actually enforces today — one cluster-wide egress deny, baseline
Pod Security on project namespaces, and no default-deny baseline yet — is in
[docs/security.md](docs/security.md), with the reasoning behind each choice in
[docs/decisions.md](docs/decisions.md).
Pod Security on project namespaces, API-server audit logs shipped off-cluster, and
no default-deny baseline yet — is in [docs/security.md](docs/security.md), with the
reasoning behind each choice in [docs/decisions.md](docs/decisions.md).

Storage is in-cluster and portable: **MinIO** for S3-compatible object storage
(backs Loki, Thanos, and backups) and **CloudNativePG** for PostgreSQL — no Azure
Expand Down Expand Up @@ -100,8 +101,11 @@ ArgoCD multi-source pattern), so upgrades are a version bump.
```
infra/ Bicep — the only Azure-specific layer
aks.bicep the cluster
keyvault.bicep durable Key Vault (separate, long-lived RG)
main.bicep orchestrator
main.bicep orchestrator (deploy this, not aks.bicep)
keyvault.bicep durable Key Vault } separate, long-lived RG;
backup-storage.bicep durable backup storage } each deployed standalone,
loganalytics.bicep durable audit workspace } so they survive a
alerts.bicep audit alerting } cluster teardown
env/webservices.bicepparam cluster params (no secrets — committed)

k8s/
Expand All @@ -127,8 +131,8 @@ The whole build is one runbook: **[docs/install.md](docs/install.md)** — from
empty subscription to a running cluster, as a single continuous shell session. It
captures every step and gotcha, ordered by dependency: prerequisites that need no
cluster first (quota, GitHub OAuth app, Key Vault, identities, secrets, backup
storage), then provisioning, then the cluster-derived wiring, then bootstrapping
ArgoCD.
storage, audit workspace + alerting), then provisioning, then the cluster-derived
wiring, then bootstrapping ArgoCD.

The common services are installed and managed **only** by ArgoCD — no manual
`helm install`. `k8s/argocd/infra-root-app.yaml` manages everything from `main`.
Expand Down Expand Up @@ -190,16 +194,20 @@ touchpoints are few and isolated:
| `disk.csi.azure.com` StorageClass | `cluster-infra/storageclass/` | Change the provisioner; keep the class names |
| LoadBalancer annotations | `traefik/values.yaml` | Provider's LB annotations, or MetalLB |
| Secrets backend (Key Vault) | External Secrets `ClusterSecretStore` | Swap the store (Vault/AWS/GCP); `ExternalSecret`s unchanged |
| Audit logs + alerting | `infra/loganalytics.bicep`, `infra/alerts.bicep`, the diagnostic setting in `aks.bicep` | No portable equivalent — the API-server audit feed is the provider's. Re-point at the new platform's log sink |

In-cluster MinIO + CloudNativePG mean **no Azure data PaaS**. The residual Azure
surface is one Bicep template, one StorageClass string, a couple of LB
annotations, and one secret-store object.
surface is the `infra/` Bicep layer (the cluster plus four standalone durable
resources), one StorageClass string, a couple of LB annotations, and one
secret-store object. Everything under `k8s/` moves unchanged.

---

## Status

The cluster and every common service have been built and validated end-to-end,
including a full **teardown + rebuild from the docs** and a full **ArgoCD
bring-up from Git**. Backups (Velero → MinIO, plus CloudNativePG's own backups)
are the next planned addition.
bring-up from Git**. Backups are in place — Velero to durable Azure Blob on two
schedules, plus CloudNativePG's own base backups and WAL archiving (see
[docs/maintenance.md](docs/maintenance.md)). What is still open is tracked under
[Not yet implemented](docs/maintenance.md#not-yet-implemented).
12 changes: 12 additions & 0 deletions docs/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,13 @@ exist until the cluster's diagnostic setting has created it — and §5b runs be
the cluster does. It could move into Bicep if the table turns out to be
pre-configurable; nobody has established that.

**Retention is not retroactive, which makes that step matter more than it looks.**
Raising retention later does not recover rows that have already aged out — they are
gone. So the §11 step is not tidying-up to be done eventually: every day it is
deferred on a running cluster silently spends a day of history the archive was
supposed to keep. On a rebuild it is durable, since the workspace outlives the
cluster and the table setting persists with it.

**What this does not give.** Attribution for the local admin certificate is still
Azure-side only — requests arrive as `masterclient` whatever the audit log records
(see [cluster-access.md](cluster-access.md)). And **Key Vault reads are not
Expand Down Expand Up @@ -323,6 +330,11 @@ paragraph above, possibly the secrets themselves. Until the query settles that,
this setting is load-bearing rather than merely conservative, and should not be
relaxed to `true` for query convenience.

Follow that through when granting access: **treat read on this workspace as read on
every secret in the cluster**, and hand it out on that basis — the same bar as a
role on the Key Vault, not the bar for a monitoring dashboard. A year of archived
rows widens that, not narrows it.

**Why a year and not 30 days.** With no detection in place, an incident will
surface incidentally — a project reports something odd, a bill looks wrong, a
credential turns up somewhere — which is routinely months, not weeks. A 30-day
Expand Down
17 changes: 13 additions & 4 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ CLUSTER_RG=webservices-v2 # resource group that will hold the cluster
LOCATION=swedencentral # Azure region for everything here

# --- Durable infrastructure (survives cluster teardown/rebuild) ---
INFRA_RG=webservices-infra # RG holding the Key Vault, identities, backup storage
INFRA_RG=webservices-infra # durable RG: Key Vault, identities, backup storage, audit workspace + alerts
KEY_VAULT_NAME=kv-scouterna-webservices # Key Vault name (globally unique, 3-24 chars)
BACKUP_STORAGE_ACCOUNT=stwsv2backup # backup storage account (globally unique, 3-24 lowercase alnum)
LOG_WORKSPACE=log-webservices # audit workspace (must match auditWorkspaceName in the bicepparam)
Expand Down Expand Up @@ -349,7 +349,8 @@ notice to a new address, and until someone acts on it the receiver exists while
delivering nothing:

```bash
az monitor action-group show -g $INFRA_RG -n audit-alerts --query "emailReceivers[].{name:name, address:emailAddress, status:status}" -o table
az monitor action-group show -g $INFRA_RG -n audit-alerts \
--query "emailReceivers[].{name:name, address:emailAddress, status:status}" -o table
```

Expect `status: Enabled`. `Disabled` means the confirmation mail was not accepted —
Expand Down Expand Up @@ -1041,11 +1042,19 @@ here will surface late and 30 days would usually have expired by then
([decisions.md](decisions.md) entry 9):

```bash
az monitor log-analytics workspace table update -g $INFRA_RG --workspace-name $LOG_WORKSPACE -n AKSAuditAdmin --retention-time 30 --total-retention-time 365
az monitor log-analytics workspace table update -g $INFRA_RG \
--workspace-name $LOG_WORKSPACE -n AKSAuditAdmin \
--retention-time 30 --total-retention-time 365

az monitor log-analytics workspace table show -g $INFRA_RG --workspace-name $LOG_WORKSPACE -n AKSAuditAdmin --query '{interactive:retentionInDays, total:totalRetentionInDays}' -o table
az monitor log-analytics workspace table show -g $INFRA_RG \
--workspace-name $LOG_WORKSPACE -n AKSAuditAdmin \
--query '{interactive:retentionInDays, total:totalRetentionInDays}' -o table
```

Do not defer this. Retention is **not retroactive**, so rows that age out before the
archive is set are gone for good — each day skipped on a running cluster is a day of
history quietly lost.

Expect `30` and `365`. A `TableNotFound` error means no audit row has landed yet —
go back to the query above; the table is created by the first event, not by §5b.
Archived rows need a search job or restore to query, not a plain `query` call.
Expand Down
2 changes: 1 addition & 1 deletion docs/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ As of the initial build:

| Component | Pin |
|---|---|
| AKS Kubernetes | 1.36.2 |
| AKS Kubernetes | 1.36 (minor alias — the patch channel owns the patch) |
| ArgoCD | v3.4.5 |
| cert-manager | v1.21.0 |
| Traefik | 41.0.2 (v3.7) |
Expand Down
4 changes: 1 addition & 3 deletions infra/env/webservices.bicepparam
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,6 @@ param vmSize = 'Standard_D4s_v6' // Intel, 4 vCPU / 16 GB, 12 data disks — dis
param nodeCount = 1 // Manual scaling: bump this + redeploy to add nodes.
param zones = ['1', '2', '3']

// --- Audit logging ---
// The workspace lives in the durable RG so audit logs outlive the cluster.
// Names, not identifiers — nothing here is a secret. docs/decisions.md entry 9.
// --- Audit logging (docs/decisions.md 9) ---
param auditWorkspaceName = 'log-webservices'
param auditWorkspaceResourceGroup = 'webservices-infra'
4 changes: 3 additions & 1 deletion infra/loganalytics.bicep
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ param workspaceName string
@description('Azure region.')
param location string = resourceGroup().location

@description('Interactive retention in days. docs/decisions.md 9.')
// The 365-day archive is set per-table in install.md §11, not here — the table
// does not exist until the cluster's diagnostic setting creates it.
@description('Interactive retention in days. Archive is separate — docs/decisions.md 9.')
@minValue(30)
@maxValue(730)
param retentionInDays int = 30
Expand Down