Skip to content

PIP-121: Introduce ServiceInfoProvider to update service info dynamically#541

Draft
BewareMyPower wants to merge 20 commits intoapache:mainfrom
BewareMyPower:bewaremypower/service-url-provider
Draft

PIP-121: Introduce ServiceInfoProvider to update service info dynamically#541
BewareMyPower wants to merge 20 commits intoapache:mainfrom
BewareMyPower:bewaremypower/service-url-provider

Conversation

@BewareMyPower
Copy link
Contributor

@BewareMyPower BewareMyPower commented Mar 2, 2026

The design is a bit different from Java client's ServiceUrlProvider because the internal classes (e.g. ClientImpl) in C++ client are not exposed unlike the Java client.

Differences:

  • The service URL, authentication and TLS trust file are coupled into a ServiceInfo struct. Otherwise, the interface could be complicated, for example, Java's AutoClusterFailoverBuilder accepts 3 methods for this purpose (secondary, secondaryAuthentication and secondaryTlsTrustCertsFilePath).
  • The ServiceUrlProvider#getServiceUrl is replaced by the getServiceInfo method added to Client.
  • The initialize method accepts a onServiceInfoUpdate callback that is used to update the service info. This can avoid adding a new public method to Client by passing an internal method of ClientImpl as the callback.
  • The probe logic should be implemented independently.

Breaking changes:

  • Since now some fields of ClientConfiguration can be modified dynamically, remove these public getters because they might return outdated values. Now it should be checked from Client::getServiceInfo.
  • The setUseTls method is removed as well because it should never be set by users. Whether to use TLS is actually determined by the service URL's scheme (e.g. pulsar+ssl:// prefix)

@BewareMyPower BewareMyPower self-assigned this Mar 2, 2026
@BewareMyPower BewareMyPower marked this pull request as draft March 2, 2026 11:49
@BewareMyPower BewareMyPower force-pushed the bewaremypower/service-url-provider branch from a9c36c8 to 7b98b25 Compare March 4, 2026 04:04
@BewareMyPower BewareMyPower force-pushed the bewaremypower/service-url-provider branch from b071d12 to a1746a9 Compare March 5, 2026 03:35
@BewareMyPower BewareMyPower changed the title PIP-121: support auto failover on client side Support connecting to a different cluster dynamically on a Client Mar 5, 2026
@BewareMyPower BewareMyPower marked this pull request as ready for review March 5, 2026 07:08
@BewareMyPower BewareMyPower marked this pull request as draft March 5, 2026 07:43
@BewareMyPower BewareMyPower marked this pull request as ready for review March 5, 2026 08:23
@BewareMyPower BewareMyPower marked this pull request as draft March 5, 2026 09:28
@BewareMyPower BewareMyPower marked this pull request as ready for review March 5, 2026 13:02
@BewareMyPower BewareMyPower modified the milestone: 4.2.0 Mar 5, 2026
@BewareMyPower BewareMyPower requested a review from Copilot March 5, 2026 13:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a public API to dynamically switch a Client instance to a different Pulsar cluster at runtime (service URL + auth + TLS trust certs), and refactors internal lookup/connection usage so in-flight components pick up the new cluster after switching.

Changes:

  • Introduce ServiceInfo, Client::updateServiceInfo(...), and Client::getServiceInfo() to support application-managed cluster failover.
  • Refactor internal components (lookup/schema/partition metadata calls) to route through ClientImpl accessors guarded by a shared mutex, so lookup service updates are visible everywhere.
  • Add/extend tests to validate cluster switching and expose token helper for reuse.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/LookupServiceTest.cc Updates tests to use ClientImpl lookup/schema accessors instead of exposing LookupService directly.
tests/ClientTest.cc Adds testUpdateServiceInfo validating switching between clusters with different auth/TLS settings.
tests/AuthTokenTest.cc Makes getToken() non-static so it can be reused by the new client test.
lib/ReaderImpl.cc Stops passing a fixed LookupService into consumers; relies on client accessors instead.
lib/PatternMultiTopicsConsumerImpl.h Updates ctor signature to remove LookupServicePtr dependency.
lib/PatternMultiTopicsConsumerImpl.cc Routes namespace topic discovery via ClientImpl so lookups reflect cluster switches.
lib/PartitionedProducerImpl.h Removes stored LookupServicePtr member.
lib/PartitionedProducerImpl.cc Fetches partition metadata via ClientImpl accessor to respect cluster switches.
lib/MultiTopicsConsumerImpl.h Removes stored LookupServicePtr member and adjusts ctors accordingly.
lib/MultiTopicsConsumerImpl.cc Fetches partition metadata via ClientImpl accessor; adds client weak-lock checks.
lib/ConsumerImpl.h Adds onClusterSwitching() hook and preserves original start message id from config.
lib/ConsumerImpl.cc Implements onClusterSwitching() to reset some consumer state when switching clusters.
lib/ConnectionPool.h Adds resetForClusterSwitching(...) to close existing conns but keep pool usable.
lib/ConnectionPool.cc Implements pool reset, closing connections with a “switch cluster” flag.
lib/ClientImpl.h Adds shared-mutex guarded lookup/schema/partition APIs + service info update/get.
lib/ClientImpl.cc Implements updateServiceInfo() and getServiceInfo(), rebuilds lookup service on switch.
lib/ClientConnection.h Extends close() signature with switchCluster flag (defaulted).
lib/ClientConnection.cc On cluster-switch close, notifies consumers via onClusterSwitching().
lib/Client.cc Exposes new public methods and routes schema calls through ClientImpl wrapper.
include/pulsar/Client.h Adds public ServiceInfo struct + Client API for updating/getting service info.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@BewareMyPower BewareMyPower changed the title Support connecting to a different cluster dynamically on a Client PIP-121: Introduce ServiceInfoProvider to update service info dynamically Mar 6, 2026
@BewareMyPower BewareMyPower marked this pull request as ready for review March 6, 2026 14:34
@BewareMyPower BewareMyPower marked this pull request as draft March 6, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants