Skip to content

Commit 5249d6f

Browse files
committed
update README and samples
1 parent 40cf6a9 commit 5249d6f

File tree

7 files changed

+186
-101
lines changed

7 files changed

+186
-101
lines changed

README.md

+142-98
Original file line numberDiff line numberDiff line change
@@ -3,21 +3,25 @@
33
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
44

55
- [Composable Operator](#composable-operator)
6-
- [Installation](#installation)
6+
- [Installation Composable](#installation-composable)
7+
- [Removing Composable](#removing-composable)
78
- [Examples](#examples)
9+
- [An example when a Kubernetes `ConfigMap` created based on a Kubernetes Service](#an-example-when-a-kubernetes-configmap-created-based-on-a-kubernetes-service)
10+
- [An example of Service.ibmcloud.ibm.com](#an-example-of-serviceibmcloudibmcom)
11+
- [getValueFrom elements](#getvaluefrom-elements)
12+
- [Format transformers](#format-transformers)
813
- [Namespaces](#namespaces)
14+
- [Deletion](#deletion)
915
- [Field path discovery](#field-path-discovery)
1016
- [Limitations](#limitations)
11-
- [Data formatting roles](#data-formatting-roles)
12-
- [Deletion](#deletion)
13-
- [TODO](#todo)
14-
- [Questions](#questions)
1517

1618
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
1719

1820
# Composable Operator
1921

20-
Composable is an overlay operator that can wrap any resource (native Kubernetes or CRD instance) and allows it to be dynamically configurable. Any field of the underlying resource can be specified with a reference to a secret or a configmap.
22+
Composable is an overlay operator that can wrap any resource (native Kubernetes or CRD instance) and allows it to be
23+
dynamically configurable. Any field of the underlying resource can be specified with a reference to any field of other
24+
Kubernetes objects.
2125

2226
The Composable Operator enables the complete declarative executable specification of collections of inter-dependent resources.
2327

@@ -39,6 +43,72 @@ curl -sL https://raw.githubusercontent.com/IBM/composable/master/hack/uninstall-
3943
```
4044
## Examples
4145

46+
Here we provide several examples of Composable usage, of course its possible usage is not restricted by the provided use
47+
cases. More other can be added later.
48+
49+
File with all examples can be found in [samples](./config/samples)
50+
51+
### An example when a Kubernetes `ConfigMap` created based on a Kubernetes Service
52+
Let's assume that we have a Kubernetes `Service`, which is part of another deployment, but we would like to create an automatic
53+
binding of our deployment objects with this `Service`. With help of `Composable` we can automatically create a `ConfigMap`
54+
with a `Service` parameter(s), e.g. the port number, whose name is `http`
55+
56+
The `Service` yaml [file](./config/samples/myService.yaml) might looks like:
57+
58+
```yaml
59+
apiVersion: v1
60+
kind: Service
61+
metadata:
62+
name: myservice
63+
namespace: default
64+
spec:
65+
sessionAffinity: None
66+
type: ClusterIP
67+
selector:
68+
app: MyApp
69+
ports:
70+
- name: http
71+
protocol: TCP
72+
port: 80
73+
targetPort: 9376
74+
```
75+
76+
The following [file](./config/samples/compCM.yaml) contains the `Composable` definition:
77+
78+
```yaml
79+
apiVersion: ibmcloud.ibm.com/v1alpha1
80+
kind: Composable
81+
metadata:
82+
name: to-cm
83+
spec:
84+
template:
85+
apiVersion: "v1"
86+
kind: ConfigMap
87+
metadata:
88+
name: myconfigmap
89+
data:
90+
servicePort:
91+
getValueFrom:
92+
kind: Service
93+
name: myservice
94+
namespace: default
95+
path: '{.spec.ports[?(@.name=="http")].port}}'
96+
format-transformers:
97+
- ToString
98+
```
99+
You can see the detail explanation of the `getValueForm` fields below, but the purpose of the object is to create a
100+
`ConfigMap` named `myconfigmap` and set `servicePort` to be equal to the port named `http` in the `Service` object named
101+
`myservice` in the `default` namespace.
102+
A Composable and a created object (`myconfigmap`) will be in teh same namespace, but input objects can be in any namespaces.
103+
104+
### An example of Service.ibmcloud.ibm.com
105+
106+
Composable-operator project works tightly with 2 other related projects: (SolSA - Solution Service Architecture)[https://github.com/IBM/solsa]
107+
and (cloud-operators)[https://github.com/IBM/cloud-operators]. The [samples](./config/samples) directory has 3 different
108+
examples of creation/configuration of `Service.ibmcloud.ibm.com` from the `cloud-opertors` project.
109+
110+
Here is one of them
111+
42112
```yaml
43113
apiVersion: ibmcloud.ibm.com/v1alpha1
44114
kind: Composable
@@ -53,42 +123,14 @@ spec:
53123
spec:
54124
instancename: mymessagehub
55125
service: Event Streams
56-
plan:
57-
getValueFrom:
58-
# any Kubernetes or CRD Kind
59-
kind: Service
60-
61-
# The discovered object name
62-
name: myservice
63-
64-
# The jsonpath style path to the field
65-
# Example: get value of nodePort from a service ports array, when the port name is "http"
66-
path: {.spec.ports[?(@.name==“http”)].port}}
67-
68-
# [Optional] the discovered object's namespace, if doesn't present, the Composable object namespace will be used
69-
# namespace: my-namespace
70-
71-
# [Optional] the desire object version, e.g. "v1", "v1alpha1", "v1beta1"
72-
# version: v1
73-
74-
# [Optional] if present, and the destination value cannot resolved, if for example a checking object does not .
75-
# or the field is not set, the default value will be used instead.
76-
# defaultValue: "value"
77-
78-
# [Optional] format-transformers, array of the available values, which are:
79-
# ToString - transforms interface to string
80-
# StringToInt - transforms string to integer
81-
# Base64ToString - decodes a base64 encoded string to a plain one
82-
# StringToBase64 - encodes a plain string to a base 64 encoded string
83-
# StringToFloat - transforms string to float
84-
# ArrayToCSString - transforms arrays of objects to a comma-separated string
85-
# StringToBool - transforms a string to boolean
86-
# JsonToObject - transforms a JSON string to an object
87-
# if presents, the operator will transform discovered data to the wished format
88-
# Example: transform data from base64 encoded string to an integer
89-
# format-transformers:
90-
# - base64ToString
91-
# - stringToInt
126+
plan:
127+
getValueFrom:
128+
kind: Secret
129+
name: mysecret
130+
path: '{.data.plan}'
131+
format-transformers:
132+
- "Base64ToString"
133+
92134
```
93135

94136
In this example, the field `plan` of the `Service.ibmcloud` instance is specified by referring to a secret. When the composable operator is created, its controller tries to read the secret and obtains the data needed for this field. If the secret is available, it then creates the `Service.ibmcloud` resource with the proper configuration. If the secret does not exist, the Composable controller keeps re-trying until it becomes available.
@@ -108,103 +150,105 @@ spec:
108150
getValueFrom:
109151
kind: ConfigMap
110152
name: myconfigmap
111-
namespace: kube-public
112-
version: v1
153+
namespace: default
113154
path: {.data.name}
114155
spec:
115156
instancename:
116157
getValueFrom:
117158
kind: ConfigMap
118159
name: myconfigmap
160+
namespace: default
119161
path: {.data.name}
120162
service: Event Streams
121163
plan:
122164
getValueFrom:
123165
kind: Secret
124166
name: mysecret
167+
namespace: default
125168
path: {.data.planKey}
126169
```
127-
128-
In the above example, the name of the underlying `Service.ibmcloud` instance is obtained from a `configmap` and the same
170+
In this example, the name of the underlying `Service.ibmcloud` instance is obtained from a `configmap` and the same
129171
name is used for the field `instancename`. This allows flexibility in defining configurations, and promotes the reuse
130172
of yamls by alleviating hard-wired information.
131173
Moreover, it can be used to configure with data that is computed dynamically as a result of the deployment of some other
132174
resource.
175+
133176
The `getValueFrom` element can point to any K8s and its extensions object. The kind of the object is defined by the`kind`
134177
element; the object name is defined by the `name` elements, and finally, the path to the data is defined by the value of
135178
the `path` element, which is a string with dots as a delimiter.
136179

137-
138-
## Namespaces
139-
140-
The `getValueFrom` section can have a field for the `namespace`. In this case, the specified namespace is used
141-
to look up the object that is being referenced. If the `namespace` field is absent then the namespace of
142-
the Composable object itself is used instead.
143-
144-
The template object should be created in the same namespaces as the `Composable` object. Therefore, we recommend do not
145-
define `namespace` in the template. If the namespace field is defined and its value does not equal to the `Composable`
146-
object namespace, no objects will be created, and `Composable` object status will contain an errro.
147-
148-
## Field path discovery
149-
150-
We use a `jsonpath` parser from `go-client` to define path to the resolving files. Here some examples:
151-
152-
* `{.data.key-name}` - returns a path to the key named `key-name` from a `ConfigMap` or from a `Secret`
153-
* `{.spec.ports[?(@.name==“http”)].port}}` - takes port value from a port named `http` from the `ports` array
154-
155-
### Limitations
156-
157-
Due to
158-
[issue #72220](https://github.com/kubernetes/kubernetes/issues/72220), `jsonpath` doesn't support regular expressions
159-
in json-path
160-
161-
## Data formatting roles
162-
163-
Frequently data retrieved from another object needs to be transformed to another format. `format-transformers` help to
164-
do it. Here are the data transformation roles:
165-
180+
## getValueFrom elements
181+
The `getValueFrom` element should be a single child of the parent element and can contain the following sub-fileds:
182+
183+
Filed | Is required | Format/Type | Comments
184+
----- | ------------|-------------|-----------------
185+
kind | Yes | String | Kind of the input object
186+
group | No | String | Defines a K8s Api group of teh checking object. Helps to resolve conflicts, when the same `Kind` defined in several groups
187+
name | Yes | String | Name of the input object
188+
namespace | No | String | Namespace of the input object, if isn't defined, the ns of the `Composable` operator will be checked
189+
path | Yes | String | The `jsonpath` formatted path to the checked filed
190+
format-transformers | No | Array of predefined strings | Used for value type transformation, see [Format transformers](#format-transformers)
191+
192+
## Format transformers
193+
194+
Sometimes, types of an input value and expected output value are not compatable, in order to resolve this issue,
195+
`Composable` supports several predefined transformers. They can be defined as a string array, so output of the a previous
196+
transformer's will be input to next one.
197+
When you define a `Composable` object, it is your responsibility to put in a correct order the transformers.
198+
199+
Currently `Composable` supports the following transformers:
200+
201+
Transformer | Transformation
202+
------------| ---------------
203+
`ToString` | returns a native string representation of any object
204+
`ArrayToCSString` | returns a comma-separated string from array's values
205+
`Base64ToString` | decodes a `base64` encoded string
206+
`StringToBase64` | encodes a string to `base64`
207+
`StringToInt` | transforms a string to an integer
208+
`StringToFloat` | transforms a string to a float
209+
`StringToBool` | transforms a string to boolean
210+
`JsonToObject` | transforms a JSON string to an object
211+
`ObjectToJson` | transforms an object to a JSON string
212+
213+
The data transformation roles are:
166214
* If there is no data transformers - original data format will be used, include complex structures such as maps or arrays.
167-
* Transformers from the data-transformers array executed one after another according to their order. Which allows
168-
creation of data transformation pipelines. For example, the following snippet defines a data transformation from a base64
215+
* Transformers from the format-transformers array executed one after another according to their order. Which allows
216+
creation of data transformation pipelines. For example, the following snippet defines transformation from a base64
169217
encoded string to a plain string and after that to integer. This transformation can be useful to retrieve data from Secrets.
170218

171219
```yaml
172220
format-transformers:
173221
- Base64ToString
174222
- StringToInt
175223
```
224+
225+
## Namespaces
226+
227+
The `getValueFrom` definition includes the destination `namespace`, the specified namespace is used
228+
to look up the referenced object. Otherwise, the `namespace` of the `Composable` object is checked.
176229

177-
* `ToString` - returns a native string representation of any object
178-
* `ArrayToCSString` - returns a comma-separated string from array's values
179-
* `Base64ToString` - decodes `base64` encoded string
180-
* `StringToBase64` - encodes a string to `base64`
181-
* `StringToInt` - transforms a string to an integer
182-
* `StringToFloat` - transforms a string to a float
183-
* `StringToBool` - transforms a string to boolean
184-
* `JsonToObject` - transforms a JSON string to an object
185-
* `ObjectToJson` - transforms an object to a JSON string
230+
The template object should be created in the same n`amespaces` as the `Composable` object. Therefore, we recommend do not
231+
define `namespace` in the template. If the namespace field is defined and its value does not equal to the `Composable`
232+
object namespace, no objects will be created, and `Composable` object status will contain an error.
186233

187234
## Deletion
188235

189236
When the Composable object is deleted, the underlying object is deleted as well.
190237
If the user deletes the underlying object manually, it is automatically recreated`.
191238

192239

240+
## Field path discovery
241+
242+
We use a `jsonpath` parser from `go-client` to define path to the resolving files. Here some examples:
193243

194-
## TODO
244+
* `{.data.key-name}` - returns a path to the key named `key-name` from a `ConfigMap` or from a `Secret`
245+
* `{.spec.ports[?(@.name==“http”)].port}}` - takes port value from a port named `http` from the `ports` array
195246

196-
* Extend e2e test framework
197-
* Allow transformers as a function, extendability
198-
* Support waiting conditions - wait when a checking resource is Online, Ready, or Running, and after that do other
199-
operations: deploy, retrieve value and deploy, start a job ...
200-
* Support resource monitoring, and automatically propagate the updates.
201-
* Replace logger
247+
### Limitations
202248

203-
## Questions
249+
Due to
250+
[issue #72220](https://github.com/kubernetes/kubernetes/issues/72220), `jsonpath` doesn't support regular expressions
251+
in json-path
204252

205-
* Should we provide a separate discovery mechanism to retrieve values from Secrets and ConfigMap.
206-
* The path can be changed from `{.Object.data.key}` to `data.key` or even just `key`.
207-
* In any case, we probably have to support transformers.`
208-
* ~~Should we eliminate the prefix `Object` in the jsonpath?~~
209253

210254

config/samples/comp1.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ spec:
1616
getValueFrom:
1717
kind: Secret
1818
name: mysecret
19+
namespace: default
1920
path: '{.data.plan}'
2021
format-transformers:
2122
- "Base64ToString"

config/samples/comp2.yaml

+5-2
Original file line numberDiff line numberDiff line change
@@ -11,21 +11,24 @@ spec:
1111
metadata:
1212
name:
1313
getValueFrom:
14-
name: myconfigmap
1514
kind: ConfigMap
15+
name: myconfigmap
16+
namespace: default
1617
path: '{.data.name}'
1718
spec:
1819
instancename:
1920
getValueFrom:
20-
name: myconfigmap
2121
kind: ConfigMap
22+
name: myconfigmap
23+
namespace: default
2224
path: '{.data.name}'
2325

2426
serviceClass: Event Streams
2527
plan:
2628
getValueFrom:
2729
kind: Secret
2830
name: mysecret
31+
namespace: default
2932
path: '{.data.plan}'
3033
format-transformers:
3134
- "Base64ToString"

config/samples/compCM.yaml

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: ibmcloud.ibm.com/v1alpha1
2+
kind: Composable
3+
metadata:
4+
name: to-cm
5+
spec:
6+
template:
7+
apiVersion: "v1"
8+
kind: ConfigMap
9+
metadata:
10+
name: myconfigmap
11+
data:
12+
servicePort:
13+
getValueFrom:
14+
kind: Service
15+
name: myservice
16+
namespace: default
17+
path: '{.spec.ports[?(@.name=="http")].port}}'
18+
format-transformers:
19+
- ToString

config/samples/myService.yaml

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
apiVersion: v1
2+
kind: Service
3+
metadata:
4+
name: myservice
5+
namespace: default
6+
spec:
7+
sessionAffinity: None
8+
type: ClusterIP
9+
selector:
10+
app: MyApp
11+
ports:
12+
- name: http
13+
protocol: TCP
14+
port: 80
15+
targetPort: 9376

config/samples/myconfigmap.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,6 @@ apiVersion: v1
22
kind: ConfigMap
33
metadata:
44
name: myconfigmap
5+
namespace: default
56
data:
67
name: mymessagehub

0 commit comments

Comments
 (0)