metadataEnsembleSize, metadataWriteQuorumSize, and metadataAckQuorumSize configurations in ManagedLedgerConfig have no effect

[geniusjoe]

Search before reporting

• I searched in the issues and found nothing similar.

Motivation

In ManagedLedgerConfig, there are three fields — metadataEnsembleSize, metadataWriteQuorumSize, and metadataAckQuorumSize — that were originally designed to allow cursor ledgers (metadata ledgers) to use different ensemble/write-quorum/ack-quorum settings than data ledgers. However, these configurations have no actual effect:

1. Getter methods are never called: getMetadataEnsembleSize(), getMetadataWriteQuorumSize(), and getMetadataAckQuorumSize() have zero callers in the entire production codebase (only their own definitions in ManagedLedgerConfig.java).

2. Setter methods are called but values are never used: BrokerService.java and ManagedLedgerWriter.java call setMetadataEnsembleSize(), setMetadataWriteQuorumSize(), and setMetadataAckQuorumSize(), but since no one reads these values, the settings are silently ignored.

3. Root cause — a regression from PR Add ledger op timeout to avoid topics stuck on ledger-creation #2535 (2018-09): In the original codebase, ManagedCursorImpl.createNewMetadataLedger() directly called bookkeeper.asyncCreateLedger(config.getMetadataEnsembleSize(), config.getMetadataWriteQuorumSize(), config.getMetadataAckQuorumSize(), ...), correctly using the metadata-specific settings. However, PR Add ledger op timeout to avoid topics stuck on ledger-creation #2535 (commit d5e88c1ec1) refactored ledger creation into a unified entry point ManagedLedgerImpl.asyncCreateLedger(), which hardcoded config.getEnsembleSize(), config.getWriteQuorumSize(), and config.getAckQuorumSize() — the data ledger settings — without distinguishing between data and metadata ledgers. Since then, cursor ledgers always use the same ensemble configuration as data ledgers, and the metadata-specific configuration fields became dead code.

4. User impact: If a user configures different ensemble sizes expecting cursor ledgers to use fewer replicas (e.g., metadataEnsembleSize=2 while data ledger uses 3), this configuration is silently ignored. Cursor ledgers will still use the data ledger's ensemble size, which is misleading.

5. Note: metadataMaxEntriesPerLedger is a separate field that is actively used by ManagedCursorImpl.shouldCloseLedger() to control cursor ledger rollover. This field is not affected.

Solution

Remove the three dead configuration fields (metadataEnsembleSize, metadataWriteQuorumSize, metadataAckQuorumSize) along with all their references. These fields have been non-functional since PR #2535 was introduced in the Pulsar 2.2 era (2018-09), meaning they have never worked in any released version of Pulsar for over 7 years.

Since no user has ever effectively used these configurations, removing them carries minimal compatibility risk. The affected files include ManagedLedgerConfig.java, BrokerService.java, ManagedLedgerWriter.java, and several test files.

Alternatives

Make metadata ensemble configurations actually work: Modify ManagedLedgerImpl.asyncCreateLedger() to accept an isMetadataLedger parameter, so that cursor ledger creation can use config.getMetadataEnsembleSize() instead of config.getEnsembleSize(). This would restore the original design intent. However, this approach requires significantly more changes:

• Adding independent configuration items in ServiceConfiguration and broker.conf for metadata ensemble settings (currently they just copy data ledger settings, so there's no way for users to set different values)
• Changing the asyncCreateLedger method signature and its callers
• This introduces additional complexity for a feature that has never been demanded in 7+ years

Moreover, cursor ledgers and data ledgers have similar availability requirements — losing a cursor ledger means consumers cannot resume from the correct position, which is just as critical as losing data itself. The original assumption that cursor ledgers could use fewer replicas to save resources doesn't hold in production, making a separate metadata ensemble configuration of marginal practical value.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!