Skip to content

Commit 27a30dd

Browse files
cluddencludden
committed
feat: adds result config to kubernetes input, improves docs
1 parent 316208a commit 27a30dd

58 files changed

Lines changed: 911 additions & 773 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 78 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
# benthos-kubernetes
22

3-
a collections of [benthos](https://github.com/Jeffail/benthos) plugins for integrating with Kubernetes.
3+
a collection of [benthos](https://github.com/Jeffail/benthos) plugins for integrating with Kubernetes.
44

5-
**Inputs:**
5+
#### Inputs
66

77
- [kubernetes](./doc/kubernetes_input.md) streams kubernetes objects for one or more configured watches
88

9-
**Outputs:**
9+
#### Outputs
1010

1111
- [kubernetes](./doc/kubernetes_output.md) creates, updates, and deleted kubernetes objects
1212
- [kubernetes_status](./doc/kubernetes_status_output.md) writes object status to kubernetes
1313

14-
**Processors:**
14+
#### Processors
1515

1616
- [kubernetes](./doc/kubernetes_processor.md) performs operations against a kubernetes cluster
1717

@@ -27,9 +27,82 @@ a collections of [benthos](https://github.com/Jeffail/benthos) plugins for integ
2727
- download a [release](https://github.com/cludden/benthos-kubernetes/releases)
2828
- as a benthos [plugin](./cmd/benthos/main.go)
2929

30+
```go
31+
package main
32+
33+
import (
34+
"github.com/Jeffail/benthos/v3/lib/service"
35+
_ "github.com/cludden/benthos-kubernetes/input"
36+
_ "github.com/cludden/benthos-kubernetes/output"
37+
_ "github.com/cludden/benthos-kubernetes/processor"
38+
)
39+
40+
func main() {
41+
service.Run()
42+
}
43+
```
44+
3045
## Getting Started
3146

32-
See [examples](./example/status.yml)
47+
```yaml
48+
input:
49+
type: kubernetes
50+
plugin:
51+
watches:
52+
- group: example.com
53+
version: v1
54+
kind: Foo
55+
owns:
56+
- group: example.com
57+
version: v1
58+
kind: Bar
59+
result:
60+
requeue: meta().exists("requeue")
61+
requeue_after: ${!meta("requeue_after").catch("")}
62+
63+
pipeline:
64+
processors:
65+
- switch:
66+
# ignore deleted items
67+
- check: meta().exists("deleted")
68+
processors:
69+
- bloblang: root = deleted()
70+
71+
# reconcile dependent resources
72+
- processors:
73+
- branch:
74+
processors:
75+
- bloblang: |
76+
apiVersion = "example.com/v1"
77+
kind = "Bar"
78+
metadata.labels = metadata.labels
79+
metadata.name = metadata.name
80+
metadata.namespace = metadata.namespace
81+
metadata.ownerReferences = [{
82+
"apiVersion": apiVersion,
83+
"kind": kind,
84+
"controller": true,
85+
"blockOwnerDeletion": true,
86+
"name": metadata.name,
87+
"uid": metadata.uid
88+
}]
89+
spec = spec
90+
- type: kubernetes
91+
plugin:
92+
operator: get
93+
- type: kubernetes
94+
plugin:
95+
operator: ${! if errored() { "create" } else { "update" } }
96+
result_map: |
97+
root.status.bar = metadata.uid
98+
root.status.status = "Ready"
99+
100+
output:
101+
type: kubernetes_status
102+
plugin: {}
103+
```
104+
105+
Additional examples provided [here](./example)
33106
34107
## License
35108

‎doc/kubernetes_input.md‎

Lines changed: 104 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
streams kubernetes objects for one or more configured watches
44

55
**Examples**
6+
67
```yaml
78
input:
89
type: kubernetes
@@ -34,61 +35,144 @@ input:
3435
owns:
3536
- version: v1
3637
kind: Pod
38+
result:
39+
requeue: meta().exists("requeue")
40+
requeue_after: ${!meta("requeue_after).or("")}
3741
```
3842
3943
## Fields
4044
41-
`group`
45+
### `result`
46+
47+
Customize the result of a reconciliation request via [synchronous responses](https://www.benthos.dev/docs/guides/sync_responses).
48+
49+
Type: `object`
4250

43-
resource group
51+
### `result.requeue`
52+
53+
A [Bloblang query](https://www.benthos.dev/docs/guides/bloblang/about/) that should return a boolean value indicating whether the input resource should be requeued. An empty string disables this functionality.
4454

4555
Type: `string`
4656
Default: `""`
4757

48-
`kind`
58+
### `result.requeue_after`
4959

50-
resource kind
60+
Specify a duration after which the input resource should be requeued. This is a string value, which allows you to customize it based on resulting payloads and their metadata using [interpolation functions](https://www.benthos.dev/docs/configuration/interpolation#bloblang-queries). An empty string disables this functionality.
5161

5262
Type: `string`
5363
Default: `""`
5464

55-
`version`
65+
### `watches[]`
66+
67+
A list of watch configurations that specify the set of kubernetes objects to target.
68+
69+
Type: `list(object)`
70+
Default: `[]`
71+
Required: `true`
5672

57-
resource version
73+
### `watches[].group`
74+
75+
Resource group selector
5876

5977
Type: `string`
6078
Default: `""`
6179

62-
`namespaces`
80+
### `watches[].kind`
81+
82+
Resource kind selector
83+
84+
Type: `string`
85+
Default: `""`
86+
Required: `true`
6387

64-
optional namespace filter
88+
### `watches[].namespaces`
6589

66-
Type: `[]string`
90+
Resource namespace selector. An empty array here indicates cluster scope.
6791

68-
`owns`
92+
Type: `list(string)`
93+
Default: `[]`
6994

70-
optional list of dependencies to watch
95+
### `watches[].owns[]`
7196

72-
Type: `[]object({group: string, version: string, kind: string })`
97+
Specifies an optional list of dependencies to watch. This requires the correct owner references to be present on the dependent objects.
7398

74-
`selector`
99+
Type: `list(object)`
100+
Default: `[]`
75101

76-
optional label selector
102+
### `watches[].owns[].group`
77103

78-
Type: `object`
104+
Dependency group selector
79105

80-
`selector.matchLabels`
106+
Type: `string`
107+
Default: `""`
108+
109+
### `watches[].owns[].kind`
110+
111+
Dependency kind selector
112+
113+
Type: `string`
114+
Default: `""`
115+
Required: `true`
116+
117+
### `watches[].owns[].version`
118+
119+
Dependency version selector
120+
121+
Type: `string`
122+
Default: `""`
123+
Required: `true`
81124

82-
optional label selector match requirements
125+
### `watches[].selector`
126+
127+
Optional label selector to apply as target filter.
83128

84129
Type: `object`
130+
Default: `{}`
131+
132+
### `watches[].selector.matchExpressions[]`
133+
134+
List of label match expressions to apply as target filter.
135+
136+
Type: `list(object)`
137+
Default: `{}`
138+
139+
### `watches[].selector.matchExpressions[].key`
140+
141+
Subject of the given expression.
142+
143+
Type: `string`
144+
Default: `""`
145+
Required: `true`
146+
147+
### `watches[].selector.matchExpressions[].operator`
148+
149+
Operator of the given expression (e.g. `Exists`, `In`, `NotIn`)
150+
151+
Type: `string`
152+
Default: `""`
153+
Required: `true`
154+
155+
### `watches[].selector.matchExpressions[].values[]`
156+
157+
List of values applied to operator in order to evaluate the expression.
158+
159+
Type: `string`
160+
Default: `[]`
85161

86-
`selector.matchExpressions`
162+
### `watches[].selector.matchLabels`
87163

88-
optional label selector match expressions
164+
Map of key value label pairs to use as target filter.
89165

90-
Type: `object({key: string, operator: string, values: []string})`
166+
Type: `map(string)`
167+
Default: `{}`
91168

169+
### `watches[].version`
170+
171+
Resource version selector
172+
173+
Type: `string`
174+
Default: `""`
175+
Required: `true`
92176

93177
## Metadata
94178

@@ -102,7 +186,3 @@ This input adds the following metadata fields to each message:
102186
- namespace
103187
- version
104188
```
105-
106-
## Synchronous Responses
107-
108-
Additionally, this input will check for a `requeue_after` metadata entry on [synchronous response](https://www.benthos.dev/docs/guides/sync_responses) messages, and if found, will requeue the object for reconciliation.

‎doc/kubernetes_output.md‎

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,39 +3,54 @@
33
creates, updates, and deleted kubernetes objects based
44

55
This output will perform the following actions for all message parts:
6+
67
- fail if the payload is not a valid kubernetes object
78
- delete the object if a `deleted` metadata key is present
89
- update the object if a `uid` is present
910
- create the object if no `uid` is present
1011

1112
**Examples**
13+
1214
```yaml
1315
input:
1416
type: kubernetes
1517
plugin:
1618
watches:
1719
- group: example.com
18-
version: v1alpha1
20+
version: v1
1921
kind: Foo
2022

2123
pipeline:
22-
processors:
23-
- bloblang: |
24-
root = match {
25-
meta().exists("deleted") => deleted()
26-
}
27-
28-
output:
29-
type: kubernetes
30-
plugin: {}
3124
processors:
3225
- bloblang: |
3326
map finalizer {
3427
root = this
3528
metadata.finalizers = metadata.finalizers.append("finalizer.foos.example.com")
3629
}
3730
root = match {
31+
meta().exists("deleted") => deleted()
3832
metadata.finalizers.or([]).contains("finalizer.foos.example.com") => deleted()
3933
_ => this.apply("finalizer")
4034
}
41-
```
35+
36+
output:
37+
type: kubernetes
38+
plugin: {}
39+
```
40+
41+
## Fields
42+
43+
### `deletion_propagation`
44+
45+
Specifies the [deletion propagation policy](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/#controlling-how-the-garbage-collector-deletes-dependents) when performing `delete` operations.
46+
47+
Type: `string`
48+
Default: `Background`
49+
Options: `Background`, `Foreground`, `Orphan`
50+
51+
### `max_in_flight`
52+
53+
The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
54+
55+
Type: `number`
56+
Default: `1`

‎doc/kubernetes_processor.md‎

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,32 @@ performs operations against a kubernetes cluster
44

55
## Fields
66

7-
`operator`
7+
### `deletion_propagation`
88

9-
specifies the kubernetes client operation to perform
9+
Specifies the [deletion propagation policy](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/#controlling-how-the-garbage-collector-deletes-dependents) used with the `delete` operator.
1010

1111
Type: `string`
12-
Options: `create`, `delete`, `get`, `update`
12+
Default: `Background`
13+
Options: `Background`, `Foreground`, `Orphan`
14+
15+
### `operator`
16+
17+
Specifies the kubernetes client operation to perform.
18+
19+
Type: `string`
20+
Options: `create`, `delete`, `get`, `status`, `update`
21+
22+
### `operator_mapping`
23+
24+
A [Bloblang mapping](https://www.benthos.dev/docs/guides/bloblang/about/) that resolves to valid operator.
25+
26+
Type: `string`
27+
28+
### `parts[]`
29+
30+
An optional array of message indexes of a batch that the processor should apply to. If left empty all messages are processed. This field is only applicable when batching messages at the input level.
31+
32+
Indexes can be negative, and if so the part will be selected from the end counting backwards starting from -1.
33+
34+
Type: `list(number)`
35+
Default: `[]`

0 commit comments

Comments
 (0)