Skip to content

Cache sticky endpoint channels#2993

Open
schlosna wants to merge 5 commits into
developfrom
davids/sticky-cache-clean
Open

Cache sticky endpoint channels#2993
schlosna wants to merge 5 commits into
developfrom
davids/sticky-cache-clean

Conversation

@schlosna
Copy link
Copy Markdown
Contributor

@schlosna schlosna commented May 15, 2026

Before this PR

Dialogue clients that make heavy use of sticky channels incur a significant construction overhead (e.g. see palantir/conjure-java#2869 ) leading to per-endpoint TypeMarker
anonymous classes, ExceptionDeserializerArgs instances, and Deserializer fields,
causing excessive allocations on high throughput services.

This adds unnecessary overhead for conjure clients that use sticky channels such as ⛰️ clients via e.g. DataServiceAsync#of(com.palantir.dialogue.EndpointChannelFactory, com.palantir.dialogue.ConjureRuntime).

See #2647 which was a basis for this PR.

After this PR

==COMMIT_MSG==

Summary

  • Introduce StickyEndpointChannelsFactory interface — replaces the Supplier<Channel> return type from StickyEndpointChannels2.create() and DialogueChannel.stickyChannels() with a typed stickyChannel() method. DialogueChannel now directly implements this interface, and DialogueChannel.stickyChannels() is deprecated in favor of stickyChannel().
  • Extract CachingEndpointChannelFactory — encapsulates the Caffeine LoadingCache<Endpoint, EndpointChannel> with delegate::endpoint as the loader into a reusable EndpointChannelFactory implementation. Cache size is configurable via Config.maxEndpointCacheSize() (default 1000), and caching is bypassed when set to <= 0.
  • Simplify StickyEndpointChannels2 internals — removes the StickyEndpointChannels2EndpointFactorySupplier and StickyEndpointChannel inner classes; queue override attachment and sticky routing are now inlined in StickyChannel2.endpoint(), reducing indirection.
  • Update ReloadingClientFactoryInternalDialogueChannel now extends StickyEndpointChannelsFactory instead of declaring its own stickyChannels() method, and callers use the new stickyChannel() API directly.

Test plan

  • Existing StickyEndpointChannels2Test passes (caching behavior is identical, just encapsulated)
  • SimulationTest passes and regenerated simulation resources reflect updated sticky channel behavior
  • ReloadingClientFactoryTest passes with the new stickyChannel() API

==COMMIT_MSG==

Possible downsides?

This encapsulates a cache of Endpoint -> EndpointChannel so there will be some memory overhead; however, use of sticky channels implies these routes intend to stick around in memory and amortize overhead for benefit of throughput.

@changelog-app
Copy link
Copy Markdown

changelog-app Bot commented May 15, 2026

Generate changelog in changelog/@unreleased

Type (Select exactly one)

  • Feature (Adding new functionality)
  • Improvement (Improving existing functionality)
  • Fix (Fixing an issue with existing functionality)
  • Break (Creating a new major version by breaking public APIs)
  • Deprecation (Removing functionality in a non-breaking way)
  • Migration (Automatically moving data/functionality to a new system)

Description

Cache sticky endpoint channels

Check the box to generate changelog(s)

  • Generate changelog entry

@schlosna schlosna force-pushed the davids/sticky-cache-clean branch 2 times, most recently from ef6dbdb to 0617986 Compare May 15, 2026 14:19
@schlosna schlosna force-pushed the davids/sticky-cache-clean branch from 0617986 to 860dd5f Compare May 15, 2026 14:21
@schlosna schlosna changed the title Davids/sticky cache clean Cache sticky endpoint channels May 15, 2026
@changelog-app
Copy link
Copy Markdown

changelog-app Bot commented May 22, 2026

Successfully generated changelog entry!

Need to regenerate?

Simply interact with the changelog bot comment again to regenerate these entries.


📋Changelog Preview

💡 Improvements

  • Cache sticky endpoint channels (#2993)

@schlosna schlosna marked this pull request as ready for review May 22, 2026 15:52
@schlosna schlosna requested review from a team, aldexis, fsamuel-bs and pkoenig10 May 22, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant