Skip to content

Commit 55c3eb0

Browse files
seansonseanson
committed
Add support for custom Persistent Volumes to enable GCS storage
1 parent 71019fe commit 55c3eb0

File tree

6 files changed

+239
-54
lines changed

6 files changed

+239
-54
lines changed

‎charts/kubernetes-agent/README.md‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,38 @@ The `main` branch will reflect the current development version of the chart. Thi
2424
| 1 | [release/kubernetes-agent/v1](https://github.com/OctopusDeploy/helm-charts/tree/release/kubernetes-agent/v1/charts/kubernetes-agent) | [here](https://github.com/OctopusDeploy/helm-charts/blob/release/kubernetes-agent/v1/charts/kubernetes-agent/README.md) | [here](https://github.com/OctopusDeploy/helm-charts/blob/release/kubernetes-agent/v1/charts/kubernetes-agent/values.yaml) |
2525

2626
### Migrations
27+
2728
Version 2 of the helm chart introduces breaking changes to `values.yaml`; some elements were renamed, while others were moved.
2829

2930
As such, upgrading from V1 to V2 of the helm chart requires user intervention.
3031

3132
This is documented [here](./migrations.md).
3233

34+
## Persistence
35+
36+
By default unless `persistence.storageClassName` has been set this chart will deploy a local NFS storage provisioner.
37+
If using your own Storage Class, please ensure that you are using a ReadWriteMany (RWX) access mode class.
38+
39+
### Custom Persistent Volumes
40+
41+
If you are using an implementation that does not use standard StorageClasses for PersistentVolumeClaim management, such as the
42+
Google Cloud's [Cloud Storage FUSE CSI driver](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-storage-fuse-csi-driver-setup)
43+
then this chart can be configured to use a custom PV.
44+
45+
As an example, using Cloud Storage FUSE CSI on Google Kubernetes Engine:
46+
47+
```yaml
48+
persistence:
49+
storageClassName: "gcs-storage"
50+
customPersistentVolume:
51+
enabled: true
52+
csi:
53+
driver: gcsfuse.csi.storage.gke.io
54+
volumeHandle: transmit-octopus-deploy-staging # GCS Bucket Name
55+
mountOptions:
56+
- implicit-dirs
57+
```
58+
3359
## Maintainers
3460
3561
| Name | Email | Url |
@@ -72,7 +98,7 @@ This is documented [here](./migrations.md).
7298
| agent.serviceAccount.name | string | Generates a name based on `agent.name` | The name of the service account for the agent pod |
7399
| agent.space | string | `"Default"` | The Space to register the agent in |
74100
| agent.tolerations | list | `[]` | The tolerations to apply to the agent pod |
75-
| agent.upgrade | object | `{"dockerAuth":{"password":"","registry":"","username":""}}` | Credentials used during agent-upgrade tasks. To be populated if encountering rate-limiting failures. |
101+
| agent.upgrade | object | `{"dockerAuth":{"password":"","registry":"","username":""}}` | Credentials used during agent-upgrade tasks. To be populated if encountering rate-limiting failures. |
76102
| agent.username | string | `""` | The username of the user used to authenticate with the target Octopus Server |
77103
| agent.usernamePasswordSecretName | string | `""` | The name of an existing Secret that contains a base64-encoded username and password for an Octopus Server user. Values must be set in `data.username` and `data.password` in secret. |
78104

@@ -109,6 +135,7 @@ This is documented [here](./migrations.md).
109135

110136
| Key | Type | Default | Description |
111137
|-----|------|---------|-------------|
138+
| persistence.customPersistentVolume.enabled | bool | `false` | If enabled, the Google Cloud Storage FUSE CSI driver will attempt to provision persistent storage via bucket. Requires persistence.storageClassName to also be set. |
112139
| persistence.nfs.affinity | object | `{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"kubernetes.io/os","operator":"In","values":["linux"]},{"key":"kubernetes.io/arch","operator":"In","values":["arm64","amd64"]}]}]}}}` | The affinities to apply to the NFS pod |
113140
| persistence.nfs.image | object | `{"pullPolicy":"IfNotPresent","repository":"octopusdeploy/nfs-server","tag":"1.0.1"}` | The repository, pullPolicy & tag to use for the NFS server |
114141
| persistence.nfs.metadata | object | `{"annotations":{},"labels":{}}` | Additional metadata to add to the NFS pod & container |
@@ -147,6 +174,8 @@ This is documented [here](./migrations.md).
147174
|-----|------|---------|-------------|
148175
| imagePullSecrets | list | `[]` | custom registry pullSecret<br> See https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod These are used for the tentacle and script pods |
149176
| nameOverride | string | `""` | Override the name of the app |
177+
| persistence.customPersistentVolume.csi | object | `{}` | |
178+
| persistence.customPersistentVolume.mountOptions | list | `[]` | |
150179

151180
----------------------------------------------
152181
Autogenerated from chart metadata using [helm-docs](https://github.com/norwoodj/helm-docs)

‎charts/kubernetes-agent/README.md.gotmpl‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,39 @@ The `main` branch will reflect the current development version of the chart. Thi
2424
| 1 | [release/kubernetes-agent/v1](https://github.com/OctopusDeploy/helm-charts/tree/release/kubernetes-agent/v1/charts/kubernetes-agent) | [here](https://github.com/OctopusDeploy/helm-charts/blob/release/kubernetes-agent/v1/charts/kubernetes-agent/README.md) | [here](https://github.com/OctopusDeploy/helm-charts/blob/release/kubernetes-agent/v1/charts/kubernetes-agent/values.yaml) |
2525

2626
### Migrations
27+
2728
Version 2 of the helm chart introduces breaking changes to `values.yaml`; some elements were renamed, while others were moved.
2829

2930
As such, upgrading from V1 to V2 of the helm chart requires user intervention.
3031

3132
This is documented [here](./migrations.md).
3233

34+
## Persistence
35+
36+
By default unless `persistence.storageClassName` has been set this chart will deploy a local NFS storage provisioner.
37+
If using your own Storage Class, please ensure that you are using a ReadWriteMany (RWX) access mode class.
38+
39+
### Custom Persistent Volumes
40+
41+
If you are using an implementation that does not use standard StorageClasses for PersistentVolumeClaim management, such as the
42+
Google Cloud's [Cloud Storage FUSE CSI driver](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-storage-fuse-csi-driver-setup)
43+
then this chart can be configured to use a custom PV.
44+
45+
As an example, using Cloud Storage FUSE CSI on Google Kubernetes Engine:
46+
47+
```yaml
48+
persistence:
49+
storageClassName: "gcs-storage"
50+
customPersistentVolume:
51+
enabled: true
52+
csi:
53+
driver: gcsfuse.csi.storage.gke.io
54+
volumeHandle: transmit-octopus-deploy-staging # GCS Bucket Name
55+
mountOptions:
56+
- implicit-dirs
57+
```
58+
59+
3360
{{ template "chart.maintainersSection" . }}
3461

3562
{{ template "chart.valuesSection" . }}

‎charts/kubernetes-agent/templates/_helpers.tpl‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,17 @@ The name of the PersistentVolumeClaim to configure
141141
{{- end }}
142142
{{- end }}
143143

144+
{{/*
145+
The name of the PersistentVolume to configure
146+
*/}}
147+
{{- define "kubernetes-agent.pvName" -}}
148+
{{- if .Values.persistence.storageClassName }}
149+
{{- printf "%s-pvc" (include "kubernetes-agent.fullName" .) }}
150+
{{- else }}
151+
{{- include "nfs.pvName" . }}
152+
{{- end }}
153+
{{- end }}
154+
144155
{{/*
145156
Turns the imagePullSecrets map into a CSV.
146157
*/}}

‎charts/kubernetes-agent/templates/custom-pv.yaml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{{- if and .Values.persistence.storageClassName .Values.persistence.customPersistentVolume.enabled }}
2+
apiVersion: v1
3+
kind: PersistentVolume
4+
metadata:
5+
annotations:
6+
pv.kubernetes.io/provisioned-by: {{ .Values.persistence.customPersistentVolume.csi.driver }}
7+
name: {{ include "kubernetes-agent.pvName" . }}
8+
labels:
9+
{{- include "kubernetes-agent.labels" . | nindent 4 }}
10+
spec:
11+
capacity:
12+
storage: {{ .Values.persistence.size }}
13+
accessModes:
14+
- ReadWriteMany
15+
persistentVolumeReclaimPolicy: Retain
16+
storageClassName: {{ .Values.persistence.storageClassName | quote }}
17+
{{- with .Values.persistence.customPersistentVolume.mountOptions }}
18+
mountOptions:
19+
{{ toYaml . | nindent 4 }}
20+
{{- end }}
21+
claimRef:
22+
name: {{ include "kubernetes-agent.pvcName" . }}
23+
namespace: {{ .Release.Namespace | quote }}
24+
{{- with .Values.persistence.customPersistentVolume.csi }}
25+
csi:
26+
{{ toYaml . | nindent 4 }}
27+
{{- end }}
28+
{{- end }}

‎charts/kubernetes-agent/tests/custom-pv_test.yaml‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
suite: "persistence"
2+
release:
3+
namespace: test-namespace
4+
templates:
5+
- templates/custom-pv.yaml
6+
tests:
7+
- it: "should match snapshot"
8+
asserts:
9+
- matchSnapshot: {}
10+
11+
- it: "is not created when storageClassName has a value and customPersistentVolume is not enabled"
12+
set:
13+
persistence:
14+
storageClassName: "value"
15+
asserts:
16+
- hasDocuments:
17+
count: 0
18+
19+
- it: "has Helm Release name in name"
20+
set:
21+
persistence:
22+
storageClassName: "value"
23+
customPersistentVolume:
24+
enabled: true
25+
asserts:
26+
- matchRegex:
27+
path: metadata.name
28+
pattern: "RELEASE-NAME"
29+
30+
- it: "set storage capacity to be persistence size"
31+
set:
32+
persistence:
33+
size: 100Gi
34+
storageClassName: "value"
35+
customPersistentVolume:
36+
enabled: true
37+
asserts:
38+
- equal:
39+
path: spec.capacity.storage
40+
value: 100Gi
41+
42+
- it: "It passes CSI configuration correctly"
43+
set:
44+
persistence:
45+
storageClassName: "value"
46+
customPersistentVolume:
47+
enabled: true
48+
csi:
49+
driver: custom.storage.k8s.io
50+
volumeHandle: foo
51+
52+
asserts:
53+
- equal:
54+
path: spec.csi.driver
55+
value: custom.storage.k8s.io
56+
- equal:
57+
path: spec.csi.volumeHandle
58+
value: foo
59+
60+
- it: "It passes mount optionscorrectly"
61+
set:
62+
persistence:
63+
storageClassName: "value"
64+
customPersistentVolume:
65+
enabled: true
66+
mountOptions:
67+
- "foo"
68+
69+
asserts:
70+
- equal:
71+
path: spec.mountOptions[0]
72+
value: foo
73+
74+
- it: "uses the correct claimRef"
75+
set:
76+
persistence:
77+
storageClassName: "value"
78+
customPersistentVolume:
79+
enabled: true
80+
asserts:
81+
- matchRegex:
82+
path: spec.claimRef.name
83+
pattern: "RELEASE-NAME"
84+
- equal:
85+
path: spec.claimRef.namespace
86+
value: test-namespace

0 commit comments

Comments
 (0)