cleanup

up

Author: mitchross

.github/instructions/argocd.instructions.md

@@ -33,18 +33,18 @@ spec:
 
 ### Three-Tier Application Discovery
 1. **Infrastructure** (`infrastructure-appset.yaml`):
-   - Paths: `infrastructure/controllers/*`, `infrastructure/storage/*`, etc.
-   - Sync wave: "1" (after ArgoCD, before apps)
-   - Creates core cluster services
+   - Explicit path list (NOT glob discovery)
+   - Sync wave: "4" (after foundation, storage, and PVC Plumber)
+   - Creates core cluster services (cert-manager, Kyverno, GPU operators, databases, gateway, etc.)
 
 2. **Monitoring** (`monitoring-appset.yaml`):
    - Paths: `monitoring/*`
-   - Sync wave: "0" (early deployment)
+   - Sync wave: "5" (after infrastructure)
    - Creates observability stack
 
 3. **Applications** (`my-apps-appset.yaml`):
    - Paths: `my-apps/*/*` (nested directories)
-   - Sync wave: "2" (after infrastructure)
+   - Sync wave: "6" (after everything else)
    - Creates user applications
 
 ### Directory-Based Discovery
@@ -91,9 +91,12 @@ retry:
 ## Sync Waves and Dependencies
 
 ### Wave Ordering
-- Wave "0": Monitoring stack (Prometheus, Grafana)
-- Wave "1": Infrastructure (Cilium, Longhorn, cert-manager)
-- Wave "2": Applications (user workloads)
+- Wave "0": Foundation (Cilium, ArgoCD, 1Password Connect, External Secrets, AppProjects)
+- Wave "1": Storage (Longhorn, Snapshot Controller, VolSync)
+- Wave "2": PVC Plumber (backup existence checker)
+- Wave "4": Infrastructure ApplicationSet (cert-manager, Kyverno, GPU operators, databases, gateway)
+- Wave "5": Monitoring ApplicationSet (Prometheus, Grafana, Loki)
+- Wave "6": My-Apps ApplicationSet (user workloads)
 
 ### CRD Handling
 For infrastructure components that install CRDs:

CLAUDE.md

@@ -809,7 +809,7 @@ kubectl exec -it gpu-pod -n app-name -- nvidia-smi
 
 - **[README.md](README.md)** - Bootstrap guide, architecture overview, and Omni/Proxmox setup
 - **[.github/copilot-instructions.md](.github/copilot-instructions.md)** - Detailed development patterns
-- **[.github/instructions/](/.github/instructions/)** - Domain-specific instructions (ArgoCD, GPU, Talos, standards)
+- **[.github/instructions/](.github/instructions/)** - Domain-specific instructions (ArgoCD, GPU, Talos, standards)
 - **[docs/pvc-plumber-full-flow.md](docs/pvc-plumber-full-flow.md)** - Complete PVC backup/restore flow from bare metal to automatic disaster recovery
 - **[docs/backup-restore.md](docs/backup-restore.md)** - Detailed backup/restore workflow with architecture diagrams
 - **[docs/network-topology.md](docs/network-topology.md)** - Network architecture details

docs/argocd.md

@@ -33,11 +33,12 @@ To solve the "chicken-and-egg" problem of bootstrapping a cluster (e.g., needing
 
 | Wave | Phase | Components | Description |
 |------|-------|------------|-------------|
-| **0** | **Foundation** | `cilium`, `1password-connect`, `external-secrets` | **Networking & Secrets**. The absolute minimum required for other pods to start and pull credentials. |
+| **0** | **Foundation** | `cilium`, `argocd`, `1password-connect`, `external-secrets`, `projects` | **Networking & Secrets**. The absolute minimum required for other pods to start and pull credentials. |
 | **1** | **Storage** | `longhorn`, `snapshot-controller`, `volsync` | **Persistence**. Depends on Wave 0 for Pod-to-Pod communication and secrets. |
-| **2** | **System** | `cert-manager`, `gpu-operator`, `databases` | **Core Services**. Depends on Storage (PVCs) and Networking (Ingress/Gateway). |
-| **3** | **Observability** | `kube-prometheus-stack`, `loki` | **Monitoring**. Monitors the healthy stack. |
-| **4** | **User** | `my-apps/*` | **Workloads**. The actual applications running on the cluster. |
+| **2** | **PVC Plumber** | `pvc-plumber` | **Backup checker**. Must be running before Kyverno policies in Wave 4 call its API. |
+| **4** | **Infrastructure** | `cert-manager`, `kyverno`, `gpu-operator`, `databases`, `gateway`, etc. | **Core Services** via ApplicationSet (explicit path list). |
+| **5** | **Monitoring** | `prometheus-stack`, `loki-stack`, `tempo` | **Observability** via ApplicationSet (discovers `monitoring/*`). |
+| **6** | **User** | `my-apps/*/*` | **Workloads** via ApplicationSet (discovers `my-apps/*/*`). |
 
 ### How It Works
 Each `Application` resource in `infrastructure/controllers/argocd/apps/` is annotated with a sync wave:

docs/network-policy.md

@@ -63,7 +63,7 @@ Located at: `infrastructure/networking/cilium/policies/block-lan-access.yaml`
 | Pod-to-Pod (cluster) | **ALLOWED** | Inter-service communication |
 | Kube-apiserver | **ALLOWED** | Kubernetes operations |
 | DNS (CoreDNS) | **ALLOWED** | Name resolution |
-| TrueNAS (specific ports) | **ALLOWED** | NFS/SMB/MinIO storage |
+| TrueNAS (specific ports) | **ALLOWED** | NFS/SMB/RustFS storage |
 | LoadBalancer IPs | **ALLOWED** | Cilium L2 announcements |
 
 ## Policy Architecture
@@ -73,7 +73,7 @@ graph TD
     subgraph "Egress Rules"
         Internet[Internet<br/>0.0.0.0/0 EXCEPT RFC1918]
         Cluster[Cluster Entities<br/>pods, nodes, apiserver]
-        Storage[Whitelisted Storage<br/>TrueNAS: ports 2049,445,9000]
+        Storage[Whitelisted Storage<br/>TrueNAS: NFS,SMB,RustFS]
         LB[LoadBalancer Pool<br/>192.168.10.32/27]
     end
 
@@ -105,7 +105,7 @@ These specific IPs are allowed on specific ports only:
 
 | IP | Hostname | Allowed Ports | Purpose |
 |----|----------|---------------|---------|
-| 192.168.10.133 | TrueNAS | 2049 (NFS), 111 (RPC), 445 (SMB), 9000 (MinIO), 30292-30293 (RustFS) | Storage backend (10G) |
+| 192.168.10.133 | TrueNAS | 2049 (NFS), 111 (RPC), 445 (SMB), 9000, 30292-30293 (RustFS S3) | Storage backend (10G) |
 | 192.168.10.46 | Wyze Bridge | 8554 (RTSP) | Camera streams for Frigate |
 | 192.168.10.14 | Proxmox | 8006 (API) | Omni/Terraform integration |
 | 192.168.10.32/27 | LB Pool | All | Cilium L2 LoadBalancer IPs |

docs/network-topology.md

@@ -53,7 +53,7 @@ The cluster uses a single network with 10G switch infrastructure:
 |--------|-----|---------|
 | Router/Gateway | 192.168.10.1 | Default route |
 | Proxmox (hp-server-1) | 192.168.10.14 | Hypervisor |
-| TrueNAS | 192.168.10.133 | NAS (NFS/SMB/MinIO S3) - 10G |
+| TrueNAS | 192.168.10.133 | NAS (NFS/SMB/RustFS S3) - 10G |
 | Control Plane 1 | 192.168.10.237 | K8s master |
 | Control Plane 2 | 192.168.10.76 | K8s master |
 | Control Plane 3 | 192.168.10.140 | K8s master |
@@ -98,8 +98,7 @@ The Cilium network policy allows these storage connections:
 |-------------|-------|---------|
 | 192.168.10.133 | 2049, 111 | NFS |
 | 192.168.10.133 | 445 | SMB |
-| 192.168.10.133 | 9000 | MinIO S3 |
-| 192.168.10.133 | 30292, 30293 | RustFS |
+| 192.168.10.133 | 9000, 30292, 30293 | RustFS S3 (Loki, Tempo, pgBackRest) |
 
 ## Troubleshooting
 

monitoring/README.md

@@ -44,7 +44,7 @@ graph TB
 - **kube-state-metrics**: Kubernetes resource metrics
 
 ### 📝 Log Aggregation (loki-stack/)
-- **Loki**: Log aggregation and storage (SingleBinary mode with filesystem storage)
+- **Loki**: Log aggregation and storage (SimpleScalable mode with RustFS S3 backend)
 - **Promtail**: Log collection agent
 - **Gateway**: HTTP access gateway