Skip to content

Commit 1de65ff

Browse files
MJarmoMJarmo
committed
add readme
Signed-off-by: MJarmo <michal.jarmolkiewicz@sap.com>
1 parent 5c89cde commit 1de65ff

File tree

1 file changed

+322
-0
lines changed

1 file changed

+322
-0
lines changed

‎openBaoSync/README.md‎

Lines changed: 322 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,322 @@
1+
# OpenBao Certificate Management for OpenTelemetry Collector
2+
3+
This setup provides automated certificate management using OpenBao (a Vault fork) with two approaches:
4+
1. **CSI Provider** - Mounts certificates as files via Kubernetes CSI volumes
5+
2. **Config Source Provider** - Injects secrets directly into collector configuration
6+
7+
## Overview
8+
9+
### CSI Provider Approach (Primary)
10+
11+
The solution uses the **OpenBao CSI Provider** which:
12+
- Automatically mounts secrets as files in pods
13+
- Optionally syncs secrets to Kubernetes secrets
14+
- Uses native Kubernetes CSI volume interface
15+
- No init containers or manual scripts needed
16+
- Supports automatic secret rotation
17+
18+
**Architecture:**
19+
```
20+
OpenBao (KV Store)
21+
↓
22+
SecretProviderClass (defines what to fetch)
23+
↓
24+
CSI Driver (mounts secrets)
25+
↓
26+
Pod Volume Mount (/mnt/secrets-store)
27+
↓
28+
Kubernetes Secret (optional sync)
29+
```
30+
31+
### Config Source Provider Approach (Alternative)
32+
33+
Uses OpenTelemetry Collector's built-in **Vault config source provider**:
34+
- Secrets injected directly into config using `${vault:path#key}` syntax
35+
- No CSI drivers required
36+
- Simpler Kubernetes setup
37+
- Only works with OpenTelemetry Collector
38+
39+
**When to use:**
40+
- ✅ Use CSI if you need certificates for multiple applications or prefer file-based access
41+
- ✅ Use Config Source if you only need secrets for Collector and want simpler setup
42+
43+
## Prerequisites
44+
45+
- Kubernetes cluster running (kind, minikube, etc.)
46+
- `kubectl` configured and working
47+
- PowerShell (for Windows) or Bash (for Linux/Mac)
48+
- **CSI Secret Store Driver** (installed automatically by setup script)
49+
- **OpenBao CSI Provider** (installed automatically by setup script)
50+
51+
## Quick Start
52+
53+
### Step 1: Deploy OpenBao
54+
55+
```powershell
56+
kubectl apply -f kubectl/openbao-deployment.yaml
57+
kubectl wait --for=condition=ready pod -l app=openbao -n openbao --timeout=120s
58+
```
59+
60+
### Step 2: Generate and Store Certificates
61+
62+
**PowerShell (Windows):**
63+
```powershell
64+
powershell -ExecutionPolicy Bypass -File scripts/setup-openbao-certs.ps1
65+
```
66+
67+
**Bash (Linux/Mac):**
68+
```bash
69+
bash scripts/setup-openbao-certs.sh
70+
```
71+
72+
This script will:
73+
1. Extract the root token from OpenBao logs
74+
2. Enable PKI secrets engine
75+
3. Generate a root CA
76+
4. Create test certificates (`test1.test.local` and `test2.test.local`)
77+
5. Store certificates in OpenBao KV store
78+
79+
### Step 3: Setup CSI Provider Integration
80+
81+
**PowerShell (Windows):**
82+
```powershell
83+
powershell -ExecutionPolicy Bypass -File scripts/setup-openbao-csi.ps1
84+
```
85+
86+
**Bash (Linux/Mac):**
87+
```bash
88+
bash scripts/setup-openbao-csi.sh
89+
```
90+
91+
This script will:
92+
1. Install CSI Secret Store Driver (if not already installed)
93+
2. Install OpenBao CSI Provider (if not already installed)
94+
3. Extract OpenBao root token and create a secret
95+
4. Apply RBAC configuration
96+
5. Create SecretProviderClass
97+
6. Update `otelcol1` deployment with CSI volumes
98+
99+
### Step 4: Setup Kubernetes Authentication (Required)
100+
101+
**Important:** The setup script configures token auth, but we use Kubernetes authentication for better security.
102+
103+
1. **Enable Kubernetes Auth in OpenBao:**
104+
```powershell
105+
$podName = kubectl get pods -n openbao -l app=openbao -o jsonpath='{.items[0].metadata.name}'
106+
$logs = kubectl logs -n openbao $podName
107+
$tokenMatch = $logs | Select-String -Pattern "Root Token:\s+(.+)" | ForEach-Object { $_.Matches.Groups[1].Value }
108+
$OPENBAO_TOKEN = $tokenMatch.Trim()
109+
kubectl exec -n openbao $podName -- sh -c "export VAULT_ADDR='http://127.0.0.1:8200' && export VAULT_TOKEN='$OPENBAO_TOKEN' && bao auth enable kubernetes"
110+
```
111+
112+
2. **Configure Kubernetes Auth:**
113+
```powershell
114+
$K8S_HOST = "https://kubernetes.default.svc"
115+
$saToken = kubectl exec -n openbao $podName -- cat /var/run/secrets/kubernetes.io/serviceaccount/token
116+
$caCert = kubectl exec -n openbao $podName -- cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
117+
kubectl exec -n openbao $podName -- sh -c "cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt > /tmp/k8s-ca.crt"
118+
kubectl exec -n openbao $podName -- sh -c "export VAULT_ADDR='http://127.0.0.1:8200' && export VAULT_TOKEN='$OPENBAO_TOKEN' && bao write auth/kubernetes/config token_reviewer_jwt='$saToken' kubernetes_host='$K8S_HOST' kubernetes_ca_cert=@/tmp/k8s-ca.crt disable_iss_validation=true"
119+
```
120+
121+
3. **Create Policy:**
122+
```powershell
123+
$policyBase64 = "cGF0aCAiY2VydHMvZGF0YS8qIiB7CiAgY2FwYWJpbGl0aWVzID0gWyJyZWFkIl0KfQ=="
124+
kubectl exec -n openbao $podName -- sh -c "export VAULT_ADDR='http://127.0.0.1:8200' && export VAULT_TOKEN='$OPENBAO_TOKEN' && echo '$policyBase64' | base64 -d | bao policy write otelcol1-policy -"
125+
```
126+
127+
4. **Create Role:**
128+
```powershell
129+
kubectl exec -n openbao $podName -- sh -c "export VAULT_ADDR='http://127.0.0.1:8200' && export VAULT_TOKEN='$OPENBAO_TOKEN' && bao write auth/kubernetes/role/otelcol1-role bound_service_account_names=otelcol1 bound_service_account_namespaces=otel-demo policies=otelcol1-policy ttl=1h"
130+
```
131+
132+
5. **Update SecretProviderClass:**
133+
```powershell
134+
kubectl apply -f kubectl/openbao-csi-secretproviderclass.yaml
135+
```
136+
137+
### Step 5: Verify
138+
139+
Check that certificates are mounted in the pod:
140+
141+
```powershell
142+
kubectl get secret otelcol1-certs -n otel-demo
143+
$podName = kubectl get pods -n otel-demo -l app=otelcol1 -o jsonpath='{.items[0].metadata.name}'
144+
kubectl exec -n otel-demo $podName -- ls -la /mnt/secrets-store/
145+
```
146+
147+
## How It Works
148+
149+
### Certificate Flow
150+
151+
Certificates are stored **ONLY in OpenBao KV Store**. When a pod starts:
152+
153+
1. **CSI Provider** authenticates to OpenBao using Kubernetes auth (ServiceAccount token)
154+
2. Fetches certificates from `certs/data/test1` in OpenBao
155+
3. Mounts them directly as files at `/mnt/secrets-store/`
156+
4. Optionally creates/updates Kubernetes secret `otelcol1-certs`
157+
158+
**Key Point:** Certificates are fetched directly from OpenBao on pod start - they are NOT stored in Kubernetes first.
159+
160+
### Certificate Locations
161+
162+
**In OpenBao KV Store:**
163+
- Path: `certs/data/test1` and `certs/data/test2`
164+
- Contains: `certificate`, `private_key`, `ca_chain`
165+
166+
**In Pod (CSI Mount):**
167+
- Mount path: `/mnt/secrets-store/`
168+
- Files: `certificate`, `private_key`, `ca_chain`
169+
170+
**In Kubernetes Secret (Synced):**
171+
- Name: `otelcol1-certs`
172+
- Namespace: `otel-demo`
173+
- Keys: `cert.crt`, `cert.key`, `ca.crt`
174+
175+
## Configuration
176+
177+
### Changing Certificate Name
178+
179+
To use a different certificate (e.g., `test2` instead of `test1`), update the SecretProviderClass:
180+
181+
```yaml
182+
objects: |
183+
- objectName: "certificate"
184+
secretPath: "certs/data/test2"
185+
secretKey: "certificate"
186+
```
187+
188+
Then apply:
189+
```powershell
190+
kubectl apply -f kubectl/openbao-csi-secretproviderclass.yaml
191+
```
192+
193+
### Authentication Methods
194+
195+
**Current Implementation (Kubernetes Auth):**
196+
- ✅ Uses Kubernetes authentication method
197+
- ✅ Pods authenticate using ServiceAccount tokens
198+
- ✅ No token storage needed (more secure)
199+
- ✅ Production-ready approach
200+
201+
**Alternative (Token Auth - Not Used):**
202+
- Uses token authentication
203+
- Token stored in Kubernetes secret `openbao-token`
204+
- Simpler setup but less secure
205+
- Suitable for development/testing only
206+
207+
## Alternative: Config Source Provider
208+
209+
For a simpler setup without CSI drivers, use the Vault Config Source Provider:
210+
211+
### Configuration
212+
213+
Add to collector config:
214+
215+
```yaml
216+
config_sources:
217+
vault:
218+
endpoint: "http://openbao.openbao.svc.cluster.local:8200"
219+
auth:
220+
method: kubernetes
221+
mount_path: "auth/kubernetes"
222+
role: "otelcol1-role"
223+
poll_interval: 5m
224+
225+
exporters:
226+
otlp:
227+
tls:
228+
cert_pem: "${vault:certs/data/test1#certificate}"
229+
key_pem: "${vault:certs/data/test1#private_key}"
230+
```
231+
232+
See `vault-config-source/` folder for complete example.
233+
234+
## File Structure
235+
236+
```
237+
openBaoSync/
238+
├── kubectl/
239+
│ ├── openbao-deployment.yaml # OpenBao deployment
240+
│ ├── openbao-csi-rbac.yaml # RBAC for CSI
241+
│ ├── openbao-csi-secretproviderclass.yaml # SecretProviderClass
242+
│ └── otelcol1-with-csi.yaml # Collector with CSI volumes
243+
├── scripts/
244+
│ ├── setup-openbao-certs.ps1/.sh # Certificate generation
245+
│ ├── setup-openbao-csi.ps1/.sh # CSI setup
246+
│ └── proof-certificate-sync.ps1 # Verification script
247+
└── vault-config-source/
248+
└── kubectl/
249+
└── otelcol1-with-vault-config-source.yaml # Config source example
250+
```
251+
252+
## Troubleshooting
253+
254+
### CSI Driver Not Installed
255+
256+
```powershell
257+
kubectl get pods -n kube-system -l app=secrets-store-csi-driver
258+
```
259+
260+
If not installed, run `setup-openbao-csi.ps1` again.
261+
262+
### Authentication Failures
263+
264+
Verify Kubernetes auth is configured:
265+
266+
```powershell
267+
$podName = kubectl get pods -n openbao -l app=openbao -o jsonpath='{.items[0].metadata.name}'
268+
$logs = kubectl logs -n openbao $podName
269+
$tokenMatch = $logs | Select-String -Pattern "Root Token:\s+(.+)" | ForEach-Object { $_.Matches.Groups[1].Value }
270+
$OPENBAO_TOKEN = $tokenMatch.Trim()
271+
kubectl exec -n openbao $podName -- sh -c "export VAULT_ADDR='http://127.0.0.1:8200' && export VAULT_TOKEN='$OPENBAO_TOKEN' && bao read auth/kubernetes/role/otelcol1-role"
272+
```
273+
274+
### Certificates Not Found
275+
276+
Verify certificates exist in OpenBao:
277+
278+
```powershell
279+
kubectl exec -n openbao $podName -- sh -c "export VAULT_ADDR='http://127.0.0.1:8200' && export VAULT_TOKEN='$OPENBAO_TOKEN' && bao kv get certs/test1"
280+
```
281+
282+
## Cleanup
283+
284+
To remove everything:
285+
286+
```powershell
287+
kubectl delete -f kubectl/otelcol1-with-csi.yaml
288+
kubectl delete -f kubectl/openbao-csi-secretproviderclass.yaml
289+
kubectl delete -f kubectl/openbao-csi-rbac.yaml
290+
kubectl delete secret openbao-token -n otel-demo 2>$null
291+
kubectl delete secret otelcol1-certs -n otel-demo 2>$null
292+
kubectl delete -f kubectl/openbao-deployment.yaml
293+
```
294+
295+
## Security Notes
296+
297+
⚠️ **Important:**
298+
- This setup uses OpenBao in **dev mode** which is **NOT suitable for production**
299+
- Root tokens are stored in pod logs (dev mode only)
300+
- **Kubernetes authentication is used** (production-ready approach)
301+
- Certificates are stored in plain text in OpenBao KV store
302+
- For production, additionally use:
303+
- Proper OpenBao deployment with seal/unseal
304+
- Encrypted storage
305+
- TLS for OpenBao communication
306+
- Proper certificate rotation
307+
308+
## Benefits
309+
310+
✅ **Native Kubernetes integration** - Uses standard CSI volume interface
311+
✅ **No init containers** - Secrets mounted directly by CSI driver
312+
✅ **Automatic rotation** - CSI driver can refresh secrets
313+
✅ **Better security** - Proper authentication methods supported
314+
✅ **Simpler configuration** - Declarative SecretProviderClass
315+
✅ **Production ready** - Follows OpenBao's official recommendations
316+
317+
## References
318+
319+
- [OpenBao CSI Provider Documentation](https://openbao.org/docs/platform/k8s/csi/examples/)
320+
- [OpenBao Kubernetes Integration](https://openbao.org/docs/platform/k8s/)
321+
- [CSI Secret Store Driver](https://secrets-store-csi-driver.sigs.k8s.io/)
322+
- [OpenTelemetry Collector Config Sources](https://opentelemetry.io/docs/collector/configuration/#config-sources)

0 commit comments

Comments
 (0)