Skip to content
This repository was archived by the owner on Mar 26, 2025. It is now read-only.

Commit c3f6f0e

Browse files
matyixbaluchicken
authored andcommitted
Readme updates
1 parent 90e12ee commit c3f6f0e

File tree

1 file changed

+40
-18
lines changed

1 file changed

+40
-18
lines changed

README.md

Lines changed: 40 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -22,33 +22,55 @@
2222

2323
# Kafka-Operator
2424

25-
The Banzai Cloud Kafka operator is a Kubernetes operator to automate provisioning, management, autoscaliging and operations of [Apache Kafka](https://kafka.apache.org) clusters deployed to K8s.
25+
The Banzai Cloud Kafka operator is a Kubernetes operator to automate provisioning, management, autoscaling and operations of [Apache Kafka](https://kafka.apache.org) clusters deployed to K8s.
2626

2727
## Overview
2828

29+
Apache Kafka is an open-source distributed streaming platform.
30+
2931
![Kafka-operator architecture](docs/img/kafka-operator-arch.png)
3032

3133
Some of the main features of the **Kafka-operator** are:
3234

3335
- Provision secure and production ready Kafka clusters
3436
- Fine grained broker configuration support
3537
- Advanced and highly configurable External Access via LoadBalancers using Envoy
36-
- Graceful Kafka cluster scaling (up and down)
38+
- Graceful Kafka cluster scaling (up and down) and rebalancing
3739
- Monitoring via Prometheus
3840
- Encrypted communication using SSL
39-
- Automatic reactiin and self healing based on alers (pluggable alerts, with meaningful defaults)
41+
- Automatic reaction and self healing based on alerts (plugin system, with meaningful default alert plugins)
4042

4143
### Motivation
4244

43-
At [Banzai Cloud](https://banzaicloud.com) we are building a Kubernetes distribution, [PKE](https://github.com/banzaicloud/pke) and platform, [Pipeline](https://github.com/banzaicloud/pipeline) and operate managed Kafka clusters for our customers.
45+
At [Banzai Cloud](https://banzaicloud.com) we are building a Kubernetes distribution, [PKE](https://github.com/banzaicloud/pke) and a hybrid-cloud container management platform, [Pipeline](https://github.com/banzaicloud/pipeline) and operate managed Kafka clusters for our customers. Apache Kafka predates Kubernetes and it has been designed mostly for `static` on-premise environments. State management, node identity, failover, etc is all part and internal to Kafka, thus making it properly work on Kubernetes and an underlying dynamic environment could be a challenge.
46+
47+
There are already several approaches to operate Kafka on Kubernetes - however we did not find any of them appropriate for a highly dynamic environment, nor satisfying our customer needs. The skillset to operate Kafka and Kubernetes often does not overlap, and these are clearly visible on the existing operators. We are lucky to have Apache Kafka and Kubernetes contributors among our team and when we decided to work on the Kafka operator we designed and PoC every step from both perspectives.
48+
49+
At the same time there is a huge interest in the Kafka community for a solution which enables Kafka on Kubernetes, both in the open source and closed source space.
4450

45-
There is a huge interest in the Kafka community for a solution which enables Kafka on Kubernetes.
46-
There were several initiatives to simplify Kafka on Kubernetes:
4751
- [Helm chart](https://github.com/confluentinc/cp-helm-charts/tree/master/charts/cp-kafka)
4852
- [Yaml files](https://github.com/Yolean/kubernetes-kafka)
4953
- [Strimzi Kafka Operator](https://github.com/strimzi/strimzi-kafka-operator)
5054

51-
However, none of these gave a full solution to **automate** the Kafka experience and make it consumable for the wider audience. Our motivation is to build an open source solution and a community which drives the innovation and features of this operator.
55+
To deep dive into more details of the most popular existing solution please see this table:
56+
57+
| | Banzai Cloud | Krallistic | Strimzi | Confluent|
58+
| ------------- | ------------ | ------------ | ------------ | ------------ |
59+
| Open source | Apache 2 | Apache 2|Apache 2| No|
60+
| Fine grained broker config support | Yes (learn more) | Limited via StatefulSet|Limited via StatefulSet|Limited via StatefulSet|
61+
| Fine grained broker volume support | Yes (learn more) | Limited via StatefulSet|Limited via StatefulSet|Limited via StatefulSet|
62+
| Monitoring | Yes | Yes|Yes|Yes|
63+
| Encryption using SSL | Yes | Yes|Yes|Yes|
64+
| Rolling Update | Work in progress | No |No|Yes|
65+
| Cluster external accesses | Envoy (single LB) | Nodeport |Nodeport or LB/broker|Yes (N/A)|
66+
| User Management via CRD | Work in progress | No |Yes|No|
67+
| Topic management via CRD | Work in progress | No |Yes|No|
68+
| Reacting to Alerts| Yes (Prometheus + Cruise Control | No |No|No|
69+
| Graceful Cluster Scaling (up and down)| Yes (using Cruise Control) | No |No|Yes|
70+
71+
**Note: this assessment was made on May 20, 2019 - if you find it inaccurate please submit a PR
72+
73+
Finally, our motivation is to build an open source solution and a community which drives the innovation and features of this operator.
5274

5375
If you are willing to kickstart your Kafka experience using Pipeline, check out the free developer beta:
5476
<p align="center">
@@ -59,17 +81,16 @@ If you are willing to kickstart your Kafka experience using Pipeline, check out
5981

6082
## Installation
6183

62-
The operator installs the 2.1.0 version of Kafka, and can run on Minikube v0.33.1+ and Kubernetes 1.12.0+.
84+
The operator installs the 2.1.0 version of Apache Kafka, and can run on Minikube v0.33.1+ and Kubernetes 1.12.0+.
6385

64-
As a pre-requisite it needs a Kubernetes cluster (you can create one using [Pipeline](https://github.com/banzaicloud/pipeline)).
65-
Also, Kafka requires Zookeeper so you need to first start a Zookeeper server if you don't already have one.
86+
As a pre-requisite it needs a Kubernetes cluster (you can create one using [Pipeline](https://github.com/banzaicloud/pipeline)). Also, Kafka requires Zookeeper so you need to first have a Zookeeper cluster if you don't already have one.
6687

67-
> Banzai Cloud's Kafka operator is a pure Kafka Operator without Zookeeper support.
88+
> We believe in the `separation of concerns` principle, thus the Kafka operator does not install nor manage Zookeeper. If you would like to have a fully automated and managed experience of Apache Kafka on Kubernetes please try it with [Pipeline](https://github.com/banzaicloud/pipeline).
6889
6990
##### Install Zookeeper
7091

71-
To install Zookeeper we recommend using [Pravega's Zookeeper Operator](https://github.com/pravega/zookeeper-operator).
72-
You can deploy Zookeeper by using a Helm chart.
92+
To install Zookeeper we recommend using the [Pravega's Zookeeper Operator](https://github.com/pravega/zookeeper-operator).
93+
You can deploy Zookeeper by using the Helm chart.
7394

7495
```bash
7596
helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com/
@@ -88,7 +109,8 @@ EOF
88109

89110
### Installation
90111

91-
> Banzai Cloud recommends to use a custom StorageClass to leverage the volume binding mode `WaitForFirstConsumer`
112+
We recommend to use a **custom StorageClass** to leverage the volume binding mode `WaitForFirstConsumer`
113+
92114
```bash
93115
apiVersion: storage.k8s.io/v1
94116
kind: StorageClass
@@ -99,10 +121,10 @@ provisioner: kubernetes.io/gce-pd
99121
reclaimPolicy: Delete
100122
volumeBindingMode: WaitForFirstConsumer
101123
```
102-
> Remember to set your Kafka cr properly to use the newly created StorageClass.
124+
> Remember to set your Kafka CR properly to use the newly created StorageClass.
103125
104126
1. Set `KUBECONFIG` pointing towards your cluster
105-
2. Run `make deploy` (deploys the operator in the `kafka` namespace to the cluster)
127+
2. Run `make deploy` (deploys the operator in the `kafka` namespace into the cluster)
106128
3. Set your Kafka configurations in a Kubernetes custom resource (sample: `config/samples/banzaicloud_v1alpha1_kafkacluster.yaml`) and run this command to deploy the Kafka components:
107129

108130
```bash
@@ -111,7 +133,7 @@ kubectl create -n kafka -f config/samples/example-secret.yaml
111133
kubectl create -n kafka -f config/samples/banzaicloud_v1alpha1_kafkacluster.yaml
112134
```
113135

114-
> In this case you have to install Prometheus with proper configuration if you want to Kafka-Operator reacting on alerts.
136+
> In this case you have to install Prometheus with proper configuration if you want the Kafka-Operator to react to alerts. Again, if you need Prometheus and would like to have a fully automated and managed experience of Apache Kafka on Kubernetes please try it with [Pipeline](https://github.com/banzaicloud/pipeline).
115137
116138

117139
### Easy way: installing with Helm
@@ -126,7 +148,7 @@ kubectl create -n kafka -f config/samples/example-secret.yaml
126148
kubectl create -n kafka -f config/samples/banzaicloud_v1alpha1_kafkacluster.yaml
127149
```
128150

129-
> In this case Prometheus will be installed and configured properly for Kafka-Operator.
151+
> In this case Prometheus will be installed and configured properly for the Kafka-Operator.
130152
131153
## Development
132154

0 commit comments

Comments
 (0)