Skip to content

Add Streamshub Console Helm charts proposal #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jankalinic wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add Streamshub Console Helm charts proposal #4

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
18a8f87
Add streamshub-console-helm-charts proposal
jankalinic Mar 5, 2026
49cf2b0
Fix validation, add entry
jankalinic Mar 5, 2026
ea7e48a
addressed comments and ideas
jankalinic Mar 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions 001-streamshub-console-helm.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Helm Installation Support for StreamsHub Console

This proposal outlines adding official Helm chart support for the StreamsHub Console operator and its Custom Resources (CRs).
This would provide a standardized installation method and lay the foundation for a broader StreamsHub Helm ecosystem.

## Current situation

There is currently no official Helm chart for the StreamsHub Console operator or its associated Custom Resources.
Among the broader StreamsHub stack, only Strimzi provides an official Helm chart.
Components such as Strimzi, Prometheus, and other dependencies each maintain their own Helm charts through their respective upstream projects.

## Motivation

Helm is widely adopted as the standard packaging mechanism for k8s applications.
A large portion of the k8s ecosystem — including GitOps platforms (e.g. Argo CD) and cloud provider marketplaces — relies on Helm as the primary installation mechanism.

By introducing an official Helm chart for the Console operator and CRs, StreamsHub could:

- **Lower the barrier to adoption** for teams already using Helm-based workflows and environments
- **Enable fine-tuned configuration** with value overrides, environment-specific configuration, and lifecycle management via Helm
- **Align with ecosystem standards**, making StreamsHub easier to discover and integrate
- **Unlock the possibility for a future StreamsHub Helm chart** — once the Console operator has its own chart, it becomes easier to compose a top-level `streamshub` chart
that depends on Console, Strimzi, Kafka, Prometheus, and other components, enabling full-stack deployment through a single `helm install` command
Copy link
Copy Markdown
Contributor

@tomncooper tomncooper Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this shouldn't be a new line?


## Proposal

The proposal is to create and maintain an official Helm chart repo for:

1. **The StreamsHub Console Operator** — managing its deployment, RBAC, service accounts, and associated configuration
2. **The Console Custom Resource (CR)** — exposing the Console instance configuration as Helm values, allowing users to customise the Console without editing raw YAML manifests
Copy link
Copy Markdown

@Frawless Frawless Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think having CR managed by helm is a good way. You will eventually end-up with endless possibilities of configs and you will need to handle different branches etc. A least in Strimzi we don't do that afaik.

What will be the usecase for managing CRs via helm? I am not using helm at all so I am just curious if there is some benefit for managing it at scale, via GitOps or something.


### Scope of initial chart(s)
Copy link
Copy Markdown

@Frawless Frawless Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mentioned CRs management to be part of charts as well, but it is not listed in the scope. It makes it quite confusing.


The initial implementation should be deliberately scoped and simple. It should cover:

- Operator's `Deployment`, `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding`
- CRD installation (either bundled or as a dependency)
- A Console CR that can be optionally deployed alongside the operator by passing a Helm value (e.g. console.instance=true), allowing users who prefer to manage the CR separately to do so

- The chart should **not** attempt to manage external dependencies such as Strimzi, Kafka, or Prometheus in this initial phase.
Those components have already well-maintained charts and could be addressed in a follow-up effort.

### Possible Future: StreamsHub Umbrella Chart
Copy link
Copy Markdown

@aswinayyolath aswinayyolath Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helm dependencies (Chart.yaml) can only point to other charts, not specific K8s CRs like a Kafka/KRaft CR...tp achieve the flow described, the umbrella chart would likely depend on the strimzi-kafka-operator chart, while the Kafka CRs themselves would be templates within the umbrella chart's templates/ directory.

Copy link
Copy Markdown

@aswinayyolath aswinayyolath Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This distinction is important for how we'll need to pass values down to the subcharts to ensure the operator is ready before the Kafka resources are reconciled

Copy link
Copy Markdown
Author

@jankalinic jankalinic Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that Chart.yaml dependencies can only reference other Helm charts, not raw Kubernetes CRs. The phrasing "declare Helm dependencies on ... " might be misleading, if that's the case i can re-phrase it. What i actually meant was:

  • The umbrella chart (if it exists in the future) declares a dependency for example on the strimzi-kafka-operator Helm chart
  • The Kafka CRs themselves live an example templates inside the umbrella chart.

Copy link
Copy Markdown
Contributor

@tomncooper tomncooper Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My personal view is that this proposal shouldn't discuss an umbrella chart in detail, that is really for a separate proposal. The only part I think is relevant to mention an umbrella chart is with respect to the CRD management and how different options relate to potentially being used as a sub-chart.


Once official charts exist for the Console operator and operands, it becomes possible to introduce a `streamshub`
umbrella Helm chart that composes the full stack. This chart would declare Helm dependencies on:
Copy link
Copy Markdown

@aswinayyolath aswinayyolath Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another thing that came to mind is that helm only processes CRDs from the top-level chart's crds/ directory. CRDs defined in subcharts are silently skipped during install. So if the umbrella chart declares streamshub-console as a subchart dependency, the Console operator's CRDs won't be installed automatically.

Copy link
Copy Markdown

@aswinayyolath aswinayyolath Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A common solution is to hoist all CRDs into the top-level umbrella chart's crds/ directory, or use a dedicated CRD loader job. Worth thinking through before the umbrella chart design is finalised, as it affects how the dependency structure would need to be laid out.

Copy link
Copy Markdown
Author

@jankalinic jankalinic Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, and one that could easily catch us out if we don't design for it right away for console (before going for the umbrella chart).

If the Console operator chart is used as a subchart of the umbrella chart, its crds/ directory will be silently ignored on install — meaning the Console CRDs won't be present when the operator tries to reconcile.

The two main options to resolve this are:

  1. Put CRDs into the umbrella chart's crds/ directory — straightforward, but means duplicating CRD definitions across charts and keeping them in sync manually
  2. Dedicated CRD loader job — a pre-install hook Job that applies CRDs via kubectl apply, giving more control but adding complexity

A possible third option worth considering is dedicated streamshub-crds sub-chart (similar to the pattern used by cert-manager chart), which can be installed independently from the operator chart. This keeps CRD lifecycle explicit and avoids silent skipping.

This might to be something we should decide and agree on before the umbrella chart design is create — it directly affects the dependency and install ordering.


- `streamshub-console` (this proposal)
- `strimzi-kafka-operator` (official Strimzi chart)
- `kafka` / `kraft` via Strimzi CRs
- `prometheus` or equivalent monitoring
- `apicurio` or equivalent schema registries
- Any other components managed by StreamsHub

This umbrella chart would give users an easy way to deploy the entire stack, significantly improving the onboarding experience and enabling catalogue listings on platforms like Artifact Hub.

### Delivery

- Charts to be hosted within the StreamsHub GitHub organisation, either in a dedicated `helm-charts` repository under a `${PROJECT_NAME}-charts/` directory
Copy link
Copy Markdown
Contributor

@tomncooper tomncooper Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My view is that this chart should be managed from within the console repository (just like Strimzi). Ideally there would be system tests that test the helm chart build with new versions of the Console. That way it will be easier to make sure the helm chart is kept up to date with new releases.

- Charts published to a Helm repository (like GitHub Pages or `oci` variants like Quay / DockerHub) + Artifact Hub following each release
- Chart versioning to follow the operator release cycle, with independent patch versioning where chart-only fixes are needed
Copy link
Copy Markdown
Contributor

@tomncooper tomncooper Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kind of my pet theory, but I think that the chart version should not be linked to the operator version, they will diverge at some point (when you need fixes in the chart only) and so you might as well start from the position that they are separate to save confusion later.

- A GitHub Actions workflow can be introduced to lint chart using helm lint on every pull request/release, catching templating errors and schema violations early
- A minimal deployment test will run on each push, spinning up a light k8s cluster (kind or minikube might be the best options) and verifying that the operator and Console CR deploy and reach a healthy state
- Chart releases will be automated using github action workflows, allowing the Console operator chart to be tested and published to the StreamsHub Helm repository

## Affected/not affected projects

**Affected:**
- `streamshub/console` — primary subject of this proposal - the operator and Console CR chart will live in or alongside this repository
- StreamsHub documentation — installation guides will need updating to include Helm-based instructions

**Not affected (at this stage):**
- Strimzi, Apicurio, Prometheus, and other stack components — remain independently managed and they will only be referenced if the future umbrella chart is taken forward

## Compatibility

- CRD management will follow Helm community best practices (CRDs in `crds/` directory) to avoid conflicts with existing CRD installations
Copy link
Copy Markdown

@aswinayyolath aswinayyolath Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great addition to the ecosystem! Regarding the CRD management strategy, since CRDs in the crds/ directory are ignored during helm upgrade, there’s a risk of version drift where hte new operator expects a schema (or a new stored version) that isn't present in the cluster.

Copy link
Copy Markdown

@aswinayyolath aswinayyolath Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given that the Kustomize proposal previously flagged this limitation, should we formalize a strategy for CRD lifecycles? Options like a dedicated crds sub-chart or a job-based upgrade hook might help ensure that the operator's reconciliation logic doesn't fail after a routine helm upgrade.

Copy link
Copy Markdown
Author

@jankalinic jankalinic Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid concern, and one that the Kustomize proposal also surfaced. Would be good to align it to stay consistent here.

Copy link
Copy Markdown
Contributor

@tomncooper tomncooper Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRD management in Helm is......interesting. Personally, I think we should stick with the same approach as Strimzi (using the /crds directory) initially. This requires specific user intervention for upgrades but that is an established pattern.

I know you have mentioned a StreamsHub umbrella chart in this proposal, but I really feel that Helm is uniquely unsuited to managing multiple operator installations. I think a far better solution would be something that allows for more fine grained control, like a meta-operator (but perhaps OLM or ansible have better options).

I do think you should have specific section on CRD management and that you should discuss the various options in relation to this chart potentially being used as a sub-chart.

- The chart will declare a minimum supported k8s version consistent with the operator's existing requirements
- Future versions of the chart will follow semantic versioning; breaking value changes will result in a major version bump and will be documented in a compatibility/migration guide
Copy link
Copy Markdown
Contributor

@tomncooper tomncooper Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said above, I think the version numbering of the chart should not be linked to the Console version, however there should be a clear mapping documented on the release page and in the documentation.


## Rejected alternatives

**Single monolithic StreamsHub chart (immediate)**
Jumping directly to a full StreamsHub umbrella chart that includes Strimzi, Kafka, and Prometheus was considered but rejected for the initial phase.
Managing other chart dependencies increases complexity and introduces coupling to third-party release cycles.
Starting with a focused Console operator chart allows us to establish chart quality and release processes before taking on a broader scope.
7 changes: 4 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ A template for new proposals can be found [here](./000-template.md).
File names should follow the format `NNN-title.md` where `NNN` is a three digit number that is incremented for each new proposal.
File name format and numbering is validated by the CI workflow defined in [validate-proposals.yml](./.github/workflows/validate-proposals.yml).

| # | Title |
|:--: |:--------------------------------------------------------------------------------|
| 000 | [StreamsHub Proposal Template](./000-template.md) |
| # | Title |
|:---:|:------------------------------------------------------------|
| 000 | [StreamsHub Proposal Template](./000-template.md) |
| 001 | [StreamsHub Console Helm](./001-streamshub-console-helm.md) |