Skip to content

Commit d14e1cf

Browse files
committed
Address reviewer comments
Signed-off-by: Thomas Cooper <code@tomcooper.dev>
1 parent fa77bbd commit d14e1cf

1 file changed

Lines changed: 28 additions & 21 deletions

File tree

001-quickstart-kustomize-repo.md

Lines changed: 28 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# StreamsHub Quick-Start Kustomize Repository
22

33
The aim of the StreamsHub organization is to support developers in creating event driven architectures.
4-
There a several complimentary open-source technologies that are often used together to provide the infrastructure for these architectures.
4+
There are several complementary open-source technologies that are often used together to provide the infrastructure for these architectures.
55
This "event stack" consists of:
66

77
- Strimzi Kafka Operator and related components (Kafka Clusters, Kafka Connect, Kafka Bridge, etc.)
@@ -10,7 +10,7 @@ This "event stack" consists of:
1010
- Kroxylicious Kafka Proxy
1111

1212
As laid out in the [current situation](#current-situation) section below, the process for installing all these components is disjointed and complex.
13-
If a developer wants to quickly assess weather this stack is appropriate for them or to test any one of the compontents as part of a wider stack they have to jump through several different hoops and customizations.
13+
If a developer wants to quickly assess whether this stack is appropriate for them or to test any one of the components as part of a wider stack they have to jump through several different hoops and customizations.
1414
What is needed is a single, opinionated installation method which deploys a minimum viable event stack.
1515

1616
This proposal recommends creating a centralized [Kustomize](https://kustomize.io/)-based repository within the StreamsHub GitHub organization that provides a unified quick-start experience for deploying the event-streaming stack on a local ephemeral (minikube, KIND, etc) or development Kubernetes cluster.
@@ -58,7 +58,7 @@ The bootstrap server address must match the deployed Kafka cluster name, if you
5858
kubectl create -f https://github.com/strimzi/strimzi-kafka-operator/raw/refs/heads/main/examples/bridge/kafka-bridge.yaml -n kafka
5959
```
6060

61-
You will then need to figure out the appropriate way to expose the bridge endpoint via Ingress or Route resources in you Kubernetes Cluster.
61+
You will then need to figure out the appropriate way to expose the bridge endpoint via Ingress or Route resources in your Kubernetes Cluster.
6262

6363
### Strimzi Kafka OAuth
6464

@@ -115,7 +115,13 @@ Kustomize is commonly used in GitOps workflows, where platform teams define bles
115115
Critically, `kubectl` can install Kustomize layers hosted remotely on GitHub, GitLab or Bitbucket, provided the Kustomize configuration files are in a repository on one of those domains.
116116
Since all StreamsHub components are hosted on GitHub, we can provide Kustomize-based development configurations that are installable directly via `kubectl` with no additional tooling.
117117

118-
This proposal recommends creating a centralized Kustomize-based repository within the StreamsHub GitHub organization that provides a unified quick-start experience for deploying the event-streaming stack on a local Kubernetes cluster.
118+
This proposal recommends creating a centralized Kustomize-based repository within the StreamsHub GitHub organization that provides a unified developer quick-start experience for deploying the event-streaming stack on a local ephemeral or remote development Kubernetes cluster.
119+
120+
It should be noted that this proposal is **not** about recommending a production deployment method.
121+
It is not about providing a mechanism to cover all possible deployment configurations, upgrade paths or security setups.
122+
This proposal is about providing the lowest resistance way to deploy a minimum viable event-streaming infrastructure stack for developers to use and assess for their needs.
123+
However, in future, we may provide a more comprehensive stack deployment method using one of the approaches discussed in the [rejected alternatives](#rejected-alternatives) section.
124+
This would, of course, be covered by a separate proposal.
119125

120126
### Prior art
121127

@@ -129,7 +135,7 @@ The quick-start repository should be organized into two main layers:
129135

130136
- Base configs (`base/`): Contains the operator deployments and CRD definitions for each component. These are the resources that must be installed first.
131137
- Stack configs (`stack/`): Contains the operand custom resources (Kafka cluster, Registry instance, Console CR, etc.) that depend on the operators being ready.
132-
- Overlays (`overlays/`): Optional variant configurations (e.g. OAuth-enabled Kafka, multi-node clusters with Cruise Control enabled) that patch the base or stack layers for different scenarios.
138+
- Overlays (`overlays/`): Optional variant configurations (e.g. multi-node clusters with Cruise Control enabled) that patch the base or stack layers for different scenarios.
133139

134140
Each layer has its own `kustomization.yaml` that references the component resources it includes.
135141
The stack layer uses remote references to pull in base configurations from the same repository.
@@ -142,13 +148,13 @@ The quick-start handles this with a two-phase install:
142148
Phase 1 — Operators and CRDs:
143149

144150
```shell
145-
kubectl apply -k 'https://github.com/streamshub/quickstart-kustomize//dev/base?ref=main'
151+
kubectl apply -k 'https://github.com/streamshub/developer-quickstart//base?ref=main'
146152
```
147153

148154
Phase 2 — Operands:
149155

150156
```shell
151-
kubectl apply -k 'https://github.com/streamshub/quickstart-kustomize//dev/stack?ref=main'
157+
kubectl apply -k 'https://github.com/streamshub/developer-quickstart//stack?ref=main'
152158
```
153159

154160
Users need to wait for the operators in Phase 1 to become ready before running Phase 2.
@@ -159,7 +165,7 @@ The `?ref=` parameter pins the installation to a specific branch or tag.
159165
To simplify the two-phase process, an optional convenience script can handle the sequencing and readiness checks automatically, as well as give richer progress information:
160166

161167
```shell
162-
curl -sL https://raw.githubusercontent.com/streamshub/quickstart-kustomize/main/dev/install.sh | bash
168+
curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | bash
163169
```
164170

165171
This script would:
@@ -189,7 +195,7 @@ All resources deployed by the quick-start should carry a common label via the Ku
189195
```yaml
190196
labels:
191197
- pairs:
192-
app.kubernetes.io/part-of: streamshub-quickstart
198+
app.kubernetes.io/part-of: streamshub-developer-quickstart
193199
includeSelectors: false
194200
```
195201
@@ -206,11 +212,11 @@ However, labels help with several key issues:
206212
- Shared-cluster detection: They allow us to query for CRs of the managed CRD types that _do not_ carry the quick-start label. If any unlabeled CRs exist for a given operator's CRD types, then we know that operator's CRDs are shared and should be retained. The uninstall script removes only that operator's Deployments, ServiceAccounts, and other non-CRD resources, while operators whose CRDs have no unlabeled CRs are fully removed including their CRDs.
207213
- Orphan discovery: After a failed or partial teardown, labels make it straightforward to find leftover quick-start resources across all namespaces:
208214
```shell
209-
kubectl get all -A -l app.kubernetes.io/part-of=streamshub-quickstart
215+
kubectl get all -A -l app.kubernetes.io/part-of=streamshub-developer-quickstart
210216
```
211217
Note that `kubectl get all` does not cover every resource type (CRDs, ClusterRoles, ClusterRoleBindings, etc.). To find labeled cluster-scoped resources, also run:
212218
```shell
213-
kubectl get crds,clusterroles,clusterrolebindings -l app.kubernetes.io/part-of=streamshub-quickstart
219+
kubectl get crds,clusterroles,clusterrolebindings -l app.kubernetes.io/part-of=streamshub-developer-quickstart
214220
```
215221
- Auditing: Labels distinguish quick-start-owned resources from independently deployed ones, which is useful for troubleshooting and capacity planning on shared clusters.
216222

@@ -220,7 +226,7 @@ Before tearing down, users on non-ephemeral (minkube, KIND, etc) or shared clust
220226
For example, to check for Strimzi-managed Kafka resources not owned by the quick-start:
221227

222228
```shell
223-
kubectl get kafkas -A --selector='!app.kubernetes.io/part-of=streamshub-quickstart'
229+
kubectl get kafkas -A --selector='!app.kubernetes.io/part-of=streamshub-developer-quickstart'
224230
```
225231

226232
Users should repeat this check for each operator group's CR types (e.g. Apicurio Registry CRs, StreamsHub Console CRs) before proceeding with Phase 2.
@@ -231,23 +237,23 @@ If no unlabeled CRs are found for any operator group, it is safe to proceed with
231237
The [uninstall script](#uninstall-script) automates these per-operator-group checks and is recommended for safe teardown.
232238

233239
For manual teardown, the process has two phases.
234-
Under most scenarios it should be safe to remove the operands (`dev/stack`) overlay:
240+
Under most scenarios it should be safe to remove the operands (`/stack`) overlay:
235241

236242
Phase 1 — Delete operands:
237243

238244
```shell
239-
kubectl delete -k 'https://github.com/streamshub/quickstart-kustomize//dev/stack?ref=main'
245+
kubectl delete -k 'https://github.com/streamshub/developer-quickstart//stack?ref=main'
240246
```
241247

242-
If the shared-resource checks above reveal no issues, users can proceed to remove the operator (`dev/base`) overlay:
248+
If the shared-resource checks above reveal no issues, users can proceed to remove the operator (`/base`) overlay:
243249

244250
Phase 2 — Delete operators and CRDs:
245251

246252
Wait for all custom resources to be fully removed before proceeding.
247253
This allows the operators to process finalizers and complete their cleanup logic.
248254

249255
```shell
250-
kubectl delete -k 'https://github.com/streamshub/quickstart-kustomize//dev/base?ref=main'
256+
kubectl delete -k 'https://github.com/streamshub/developer-quickstart//base?ref=main'
251257
```
252258

253259
Running Phase 2 before Phase 1 completes risks finalizer stalling and, on shared clusters, cascade-deleting CRs that belong to other deployments.
@@ -257,7 +263,7 @@ Running Phase 2 before Phase 1 completes risks finalizer stalling and, on shared
257263
To complement the `install.sh` convenience script, an `uninstall.sh` script should be provided that automates the sequencing and safety checks:
258264

259265
```shell
260-
curl -sL https://raw.githubusercontent.com/streamshub/quickstart-kustomize/main/dev/uninstall.sh | bash
266+
curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | bash
261267
```
262268

263269
This script would:
@@ -290,26 +296,27 @@ The initial quick-start should deploy the following components:
290296

291297
Kustomize overlays make it straightforward to add new configurations without modifying the base stack:
292298

293-
- OAuth overlay: An overlay that patches the Kafka cluster to enable OAuth authentication, potentially using the Keycloak operator for automated realm configuration
294299
- Kroxylicious overlay: An overlay that adds the Kroxylicious Kafka Proxy to the stack
295300
- Namespace customization: Overlays or patches that allow deploying into a user-specified namespace
296301
- Kafka Bridge overlay: An overlay that adds the Strimzi Kafka Bridge for HTTP-based Kafka access
297302
- Prometheus overlay: An overlay that deploys Prometheus and configures metrics collection from the stack components, enabling the StreamsHub Console's metrics dashboards
298303

299-
A _vanity URL_ (e.g. `streamshub.io/install/dev`) could be a future enhancement layered on top of the Kustomize repository. The vanity URL backend could use Kustomize internally to generate the appropriate YAML, accepting parameters for namespace customization and component selection. This would complement rather than replace the direct GitHub-based Kustomize approach.
304+
A _vanity URL_ (e.g. `streamshub.io/install/dev`) could be a future enhancement layered on top of the Kustomize repository.
305+
The vanity URL backend could use Kustomize internally to generate the appropriate YAML, accepting parameters for namespace customization and component selection.
306+
This would complement rather than replace the direct GitHub-based Kustomize approach.
300307

301308
### Considerations
302309

303310
- De-facto platform: By creating a combined dev-stack installation, we are creating a de-facto "platform" that will need to be tested for compatibility across component releases. Changes to any upstream component could break the stack, so integration testing will be needed.
304311
- Dev-to-production risk: An opinionated dev-stack install runs the risk of being adopted for production use. The quick-start should include clear documentation that this is a development-only configuration. Resource limits, security configurations and storage settings are not suitable for production.
305-
- OAuth complexity: Strimzi Kafka OAuth has many possible configurations and security-critical settings. A simple quick-start for OAuth-enabled setups may not be advisable as an initial target, but could be explored as an overlay once the base stack is stable.
312+
- OAuth complexity: Strimzi Kafka OAuth has many possible configurations and security-critical settings. A simple quick-start for OAuth-enabled setups may not be advisable as an initial target. It would be prudent to leave this to a more robust deployment system, one that can handle production level configurations.
306313
- CRD deletion on shared clusters: CRD deletion is a cluster-scoped operation that cascade-deletes all custom resources of that type across every namespace. The uninstall script will group CRDs by their parent operator and check each group independently for non-quick-start custom resources. If any CRD in an operator's group has unlabeled CRs, all CRDs for that operator are retained and only non-CRD resources are removed. Operator groups with no shared CRD usage will be fully cleaned up including CRD deletion.
307314

308315
## Affected/not affected projects
309316

310317
### Affected
311318

312-
- quickstart-kustomize (new repository): A new repository to be created in the StreamsHub GitHub organization containing all Kustomize configurations, the install script and documentation.
319+
- developer-quickstart (new repository): A new repository to be created in the StreamsHub GitHub organization containing all Kustomize configurations, the install script and documentation.
313320
- StreamsHub Console: The Console operator and CR configurations will be referenced in the quick-start. A Console CR configuration that works without authenticated Kafka listeners or Prometheus will need to be defined.
314321

315322
### Not affected

0 commit comments

Comments
 (0)