Skip to content

feat(kafka): update existing topic partitions and configs during provisioning#2095

Merged
Mokto merged 1 commit intosentry-kubernetes:developfrom
bhataprameya:Kafka-Topic-updates
Mar 23, 2026
Merged

feat(kafka): update existing topic partitions and configs during provisioning#2095
Mokto merged 1 commit intosentry-kubernetes:developfrom
bhataprameya:Kafka-Topic-updates

Conversation

@bhataprameya
Copy link
Copy Markdown
Contributor

What

Extends the external Kafka provisioning Job to update existing topics, not just
create them. After the initial --create --if-not-exists pass, a second pass now:

  • Increases partition counts when the desired value in values.yaml exceeds the
    live topic's current partition count.
  • Applies topic-level config changes (e.g. retention.ms, cleanup.policy,
    min.compaction.lag.ms) when the live config differs from the desired state.

Why

Previously, the provisioning Job was create-only — if a topic already existed, any
configuration drift (partition count, cleanup policy, compaction lag, etc.) was
silently ignored. Operators had to manually alter topics via Kafka CLI. This change
makes the Job idempotent and convergent: it drives topics toward the desired state
on every helm upgrade.

Details

  • Partition increases use kafka-topics.sh --alter --partitions.
  • Detection is done by comparing --describe output against desired values; updates
    are skipped when already in sync.

@Mokto Mokto merged commit 203d17e into sentry-kubernetes:develop Mar 23, 2026
2 checks passed
@Mokto Mokto mentioned this pull request Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants