Skip to content

Support namespace bundle count (bundles) on pulsar_namespace #231

Description

@david-streamlio

Summary

pulsar_namespace cannot express a namespace's bundle count. The resource has no bundles argument, so the number of bundles a namespace is created with — a value that materially affects load distribution and cannot be lowered later — is unmanageable from Terraform.

Evidence

The full pulsar_namespace schema, including the nested namespace_config block, has no bundle attribute:

actions, anti_affinity, backlog_quota, bookkeeper_ack_quorum, bookkeeper_ensemble,
bookkeeper_write_quorum, delete_mode, dispatch_byte_throttling_rate,
dispatch_msg_throttling_rate, dispatch_rate, enable, enable_deduplication,
enable_delete_while_inactive, inactive_topic, is_allow_auto_update_schema,
managed_ledger_max_mark_delete_rate, max_consumers_per_subscription,
max_consumers_per_topic, max_inactive_duration, max_producers_per_topic,
message_ttl_seconds, namespace, namespace_config, offload_threshold_size_in_mb,
partitions, permission_grant, persistence_policies, rate_period_seconds,
replication_clusters, retention_minutes, retention_policies, retention_size_in_mb,
role, schema_auto_update_compatibility_strategy, schema_compatibility_strategy,
schema_validation_enforce, subscription_dispatch_rate,
subscription_expiration_time_minutes, tenant, topic_auto_creation

The only bundle-related type in the repo is types.SplitNS (types/types.go:41-44), which is not wired to any registered resource:

SplitNS struct {
	Bundle             string
	UnloadSplitBundles bool
}

This is genuinely namespace-scoped

Worth stating explicitly, because most bundle tuning is not. Checked against a live 4.x cluster:

  • Namespace-scoped: the bundle count. It appears in namespaces policies <ns> as bundles.numBundles, and is settable at creation via namespaces create --bundles N.
  • Broker/cluster-wide only: every split thresholdloadBalancerAutoBundleSplitEnabled, loadBalancerNamespaceBundleMaxTopics, ...MaxSessions, ...MaxMsgRate, ...MaxBandwidthMbytes, loadBalancerNamespaceMaximumBundles, ...SplitConditionHitCountThreshold, defaultNamespaceBundleSplitAlgorithm. pulsar-admin namespaces exposes only bundles (list) and split-bundle (manual), with no per-namespace threshold command, and none of these keys appear among the namespace policy fields.

So this request is deliberately limited to the count. The thresholds belong in broker configuration and are out of scope for this provider.

Why it matters

Bundle count is a one-way door: Pulsar can only increase it. A namespace created with the default 4 and later needing 32 requires either split-bundle operations or recreation — so the value chosen at creation has lasting consequences, and today it is invisible to Terraform.

In practice this shows up when tuning load distribution: raising a namespace from 4 to 32 bundles is a routine remediation for broker imbalance, and it currently has to be applied out of band and then remembered. A cluster rebuilt from Terraform silently comes back at the default.

Proposal

Add an optional, ForceNew bundles attribute to pulsar_namespace:

resource "pulsar_namespace" "example" {
  tenant    = "my-tenant"
  namespace = "my-namespace"
  bundles   = 32
}
  • Type: schema.TypeInt, Optional: true, Computed: true — the broker picks a default (defaultNumberOfNamespaceBundles, normally 4) when unset, and a server-chosen value must not read as drift.
  • ForceNew: true, since the count cannot be reduced in place. An increase could in principle be handled with split-bundle, but treating any change as a replacement is the honest first cut; in-place increase can follow separately if wanted.
  • Validation: positive integer, power of two, matching what the admin API accepts.

Happy to open a PR if the shape looks right.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions