Skip to content

Commit 791edc8

Browse files
dee-kryvenkodee-kryvenko
authored
Init (#1)
Init
1 parent aa3fe2c commit 791edc8

File tree

17 files changed

+1772
-2
lines changed

17 files changed

+1772
-2
lines changed

‎.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,5 @@
1313

1414
# Dependency directories (remove the comment below to include it)
1515
# vendor/
16+
17+
bin/

‎README.md

+210-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,210 @@
1-
# kubernetes-pvc-provisioner
2-
Dynamic PVC provisioner for pods requests via annotations
1+
# kubernetes-dynamic-reclaimable-pvc-controllers
2+
3+
Dynamic PVC provisioner for pods requesting it via annotations. Automatic PV releaser.
4+
5+
## Features
6+
7+
- PVC Provisioner
8+
- Dynamically create PVC for Pods requesting it via the annotations.
9+
- Pod is automatically set as `ownerReferences` to the PVC - guaranteeing its deletions upon Pod deletion.
10+
- PV Releaser
11+
- Automatically associates Releaser with PVs claimed by PVCs that were created by Provisioner with the same `--controller-id`.
12+
- Deletes `claimRef` from PVs associated with Releaser to move their status from `Released` to `Available` **without deleting any data**.
13+
- Provisioner and Releaser are two separate controllers under one roof, and they can be deployed separately.
14+
- You can use Provisioner alone for something like Jenkins Kubernetes plugin that doesn't allow PVC creation on its own and automate PVC provisioning from the pod requests. Provisioner on its own will not make PVs automatically reclaimable.
15+
- You can use Releaser alone - provided you associate either your PVCs or PVs with it on your own. That will set PVCs able to automatically reclaim associated PVs with whatever data left in it from previous consumer.
16+
- To make Releaser and Deployer work together - they need to have the same `--controller-id`.
17+
18+
## Disclaimers
19+
20+
**Provisioner Controller ignores RBAC. If the user creating the Pod didn't had permissions to create PVC - it will still be created as long as Provisioner has access to do it.**
21+
22+
**Releaser Controller is by design automatically makes PVs with `reclaimPolicy: Retain` available to be reclaimed by other consumers without cleaning up any data. Use it with caution - this behavior might not be desirable in most cases. Any data left on the PV after the previous consumer will be available to all the following consumers. You may want to use StatefulSets instead. This controller might be ideal for something like build cache - insensitive data by design required to be shared among different consumers. There is many use cases for this, one of them is documented in [examples/jenkins-kubernetes-plugin-with-build-cache](examples/jenkins-kubernetes-plugin-with-build-cache).**
23+
24+
## PVC Provisioner Controller
25+
26+
### Provision
27+
28+
Pods can request PVC to be automatically created for it via the annotation:
29+
30+
```yaml
31+
apiVersion: v1
32+
kind: Pod
33+
metadata:
34+
name: pod-with-dynamic-reclaimable-pvc
35+
annotations:
36+
dynamic-pvc-provisioner.kubernetes.io/<volumeName>.enabled: "true"
37+
dynamic-pvc-provisioner.kubernetes.io/<volumeName>.pvc: |
38+
apiVersion: v1
39+
kind: PersistentVolumeClaim
40+
spec:
41+
storageClassName: reclaimable-storage-class
42+
resources:
43+
requests:
44+
storage: 1Gi
45+
accessModes:
46+
- ReadWriteOnce
47+
spec:
48+
volumes:
49+
- name: <volumeName>
50+
persistentVolumeClaim:
51+
claimName: reclaimable-pvc
52+
containers:
53+
- name: web
54+
image: nginx
55+
volumeMounts:
56+
- name: <volumeName>
57+
mountPath: /data
58+
```
59+
60+
Provisioner listens for pods created/updated with `dynamic-pvc-provisioner.kubernetes.io/*` annotations.
61+
The following conditions must be met in order for provisioner to create requested PVC:
62+
63+
- `dynamic-pvc-provisioner.kubernetes.io/<volumeName>.enabled` must be `true`.
64+
- `dynamic-pvc-provisioner.kubernetes.io/<volumeName>.pvc` must be set to a valid yaml or json representing a single `PersistentVolumeClaim` object.
65+
- `spec.volumes[].name` with that name must exist and have `spec.volumes[].persistentVolumeClaim` on it.
66+
- `spec.volumes[].persistentVolumeClaim.claimName` must not already exist.
67+
- `status.phase` must be `Pending`
68+
69+
If all these conditions are met - this controller will automatically create a PVC as defined in `dynamic-pvc-provisioner.kubernetes.io/<volumeName>.pvc`.
70+
71+
Provisioner will apply following modifications to the `dynamic-pvc-provisioner.kubernetes.io/<volumeName>.pvc` before creating an actual PVC:
72+
73+
- Original `metadata.name` will be ignored and set to `spec.volumes[].persistentVolumeClaim.claimName` from matching `spec.volumes[].name` to `<volumeName>`. `metadata.name` not required to be set in `dynamic-pvc-provisioner.kubernetes.io/<volumeName>.pvc`.
74+
- `metadata.labels."dynamic-pvc-provisioner.kubernetes.io/managed-by"` will be set to refer to the current Controller ID.
75+
- `metadata.ownerReferences` will be set referring to the current pod as an owner - guaranteeing PVC to be deleted when the pod is deleted.
76+
77+
### Usage
78+
79+
```
80+
Usage of dynamic-pvc-provisioner:
81+
-add_dir_header
82+
If true, adds the file directory to the header of the log messages
83+
-alsologtostderr
84+
log to standard error as well as files
85+
-controller-id string
86+
this controller identity name - use the same string for both provisioner and releaser
87+
-kubeconfig string
88+
optional, absolute path to the kubeconfig file
89+
-lease-lock-id string
90+
optional, the lease lock holder identity name (default <computed>)
91+
-lease-lock-name string
92+
the lease lock resource name
93+
-lease-lock-namespace string
94+
optional, the lease lock resource namespace; default to -namespace
95+
-log_backtrace_at value
96+
when logging hits line file:N, emit a stack trace
97+
-log_dir string
98+
If non-empty, write log files in this directory
99+
-log_file string
100+
If non-empty, use this log file
101+
-log_file_max_size uint
102+
Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
103+
-logtostderr
104+
log to standard error instead of files (default true)
105+
-namespace string
106+
limit to a specific namespace - only for provisioner
107+
-one_output
108+
If true, only write logs to their native severity level (vs also writing to each lower severity level)
109+
-skip_headers
110+
If true, avoid header prefixes in the log messages
111+
-skip_log_headers
112+
If true, avoid headers when opening log files
113+
-stderrthreshold value
114+
logs at or above this threshold go to stderr (default 2)
115+
-v value
116+
number for the log level verbosity
117+
-vmodule value
118+
comma-separated list of pattern=N settings for file-filtered logging
119+
```
120+
121+
Example:
122+
123+
```
124+
dynamic-pvc-provisioner \
125+
-controller-id reclaimable-pvc-test \
126+
-namespace default \
127+
-lease-lock-name reclaimable-pvc-provisioner-test
128+
```
129+
130+
## PV Releaser Controller
131+
132+
For Releaser to be able to make PVs claimed by Provisioner `Available` after PVC is gone - Releaser and Provisioner must share the same Controller ID.
133+
134+
### Associate
135+
136+
Once `Released` - PVs doesn't have any indication that they were once associated with a PVC that had association with this Controller ID. To establish this relation - we must catch it while PVC still exists and mark it with our label. If Releaser was down the whole time PVC existed - PV could never be associated making it now orphaned and it will stay as `Released` - Releaser can't know it have to make it `Available`.
137+
138+
Releaser listens for PV creations/updates.
139+
The following conditions must be met for a PV to be associated with a Releaser:
140+
141+
- PV doesn't already have `metadata.labels."reclaimable-pv-releaser.kubernetes.io/managed-by"` association.
142+
- `spec.claimRef` must refer to a PVC that has `metadata.labels."dynamic-pvc-provisioner.kubernetes.io/managed-by"` set to this Controller ID.
143+
- `--disable-automatic-association` must be `false`.
144+
145+
To establish association Releaser will set itself to `metadata.labels."reclaimable-pv-releaser.kubernetes.io/managed-by"` on this PV.
146+
147+
### Release
148+
149+
Releaser watches for PVs to be released.
150+
The following conditions must be met for a PV to be made `Available`:
151+
152+
- `metadata.labels."reclaimable-pv-releaser.kubernetes.io/managed-by"` must be set to this Controller ID.
153+
- `status.phase` must be `Released`.
154+
155+
If these conditions are met, Releaser will set `spec.claimRef` to `null`. That will make Kubernetes eventually to mark `status.phase` of this PV as `Available` - making other PVCs able to reclaim this PV. Releaser will also delete `metadata.labels."reclaimable-pv-releaser.kubernetes.io/managed-by"` to remove association - the next PVC might be managed by something else.
156+
157+
### Usage
158+
159+
```
160+
Usage of reclaimable-pv-releaser:
161+
-add_dir_header
162+
If true, adds the file directory to the header of the log messages
163+
-alsologtostderr
164+
log to standard error as well as files
165+
-controller-id string
166+
this controller identity name - use the same string for both provisioner and releaser
167+
-disable-automatic-association
168+
disable automatic PV association
169+
-kubeconfig string
170+
optional, absolute path to the kubeconfig file
171+
-lease-lock-id string
172+
optional, the lease lock holder identity name (default <computed>)
173+
-lease-lock-name string
174+
the lease lock resource name
175+
-lease-lock-namespace string
176+
optional, the lease lock resource namespace; default to -namespace
177+
-log_backtrace_at value
178+
when logging hits line file:N, emit a stack trace
179+
-log_dir string
180+
If non-empty, write log files in this directory
181+
-log_file string
182+
If non-empty, use this log file
183+
-log_file_max_size uint
184+
Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
185+
-logtostderr
186+
log to standard error instead of files (default true)
187+
-namespace string
188+
limit to a specific namespace - only for provisioner
189+
-one_output
190+
If true, only write logs to their native severity level (vs also writing to each lower severity level)
191+
-skip_headers
192+
If true, avoid header prefixes in the log messages
193+
-skip_log_headers
194+
If true, avoid headers when opening log files
195+
-stderrthreshold value
196+
logs at or above this threshold go to stderr (default 2)
197+
-v value
198+
number for the log level verbosity
199+
-vmodule value
200+
comma-separated list of pattern=N settings for file-filtered logging
201+
```
202+
203+
Example:
204+
205+
```
206+
reclaimable-pv-releaser \
207+
-controller-id reclaimable-pvc-test \
208+
-lease-lock-name reclaimable-pvc-releaser-test \
209+
-lease-lock-namespace default
210+
```

‎cmd/dynamic-pvc-provisioner/main.go

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
package main
2+
3+
import (
4+
"context"
5+
controller "github.com/plumber-cd/kubernetes-dynamic-reclaimable-pvc-controllers"
6+
"github.com/plumber-cd/kubernetes-dynamic-reclaimable-pvc-controllers/provisioner"
7+
clientset "k8s.io/client-go/kubernetes"
8+
"k8s.io/client-go/rest"
9+
klog "k8s.io/klog/v2"
10+
)
11+
12+
func main() {
13+
var c controller.Controller
14+
run := func(
15+
ctx context.Context,
16+
stopCh <-chan struct{},
17+
config *rest.Config,
18+
client *clientset.Clientset,
19+
namespace string,
20+
controllerId string,
21+
) {
22+
c = provisioner.New(ctx, client, namespace, controllerId)
23+
if err := c.Run(2, stopCh); err != nil {
24+
klog.Fatalf("Error running provisioner: %s", err.Error())
25+
}
26+
}
27+
stop := func(config *rest.Config, client *clientset.Clientset) {
28+
if c != nil {
29+
c.Stop()
30+
}
31+
}
32+
controller.Main(run, stop)
33+
}

‎cmd/reclaimable-pv-releaser/main.go

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"flag"
6+
controller "github.com/plumber-cd/kubernetes-dynamic-reclaimable-pvc-controllers"
7+
"github.com/plumber-cd/kubernetes-dynamic-reclaimable-pvc-controllers/releaser"
8+
clientset "k8s.io/client-go/kubernetes"
9+
"k8s.io/client-go/rest"
10+
klog "k8s.io/klog/v2"
11+
)
12+
13+
func main() {
14+
var disableAutomaticAssociation bool
15+
flag.BoolVar(&disableAutomaticAssociation, "disable-automatic-association", false, "disable automatic PV association")
16+
17+
var c controller.Controller
18+
run := func(
19+
ctx context.Context,
20+
stopCh <-chan struct{},
21+
config *rest.Config,
22+
client *clientset.Clientset,
23+
namespace string,
24+
controllerId string,
25+
) {
26+
c = releaser.New(ctx, client, namespace, controllerId, disableAutomaticAssociation)
27+
if err := c.Run(2, stopCh); err != nil {
28+
klog.Fatalf("Error running releaser: %s", err.Error())
29+
}
30+
}
31+
stop := func(config *rest.Config, client *clientset.Clientset) {
32+
if c != nil {
33+
c.Stop()
34+
}
35+
}
36+
controller.Main(run, stop)
37+
}

0 commit comments

Comments
 (0)