Feat: Add ZK config cache - #3612
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #3612 +/- ##
===========================================
+ Coverage 46.76% 54.88% +8.11%
===========================================
Files 295 476 +181
Lines 17172 37223 +20051
===========================================
+ Hits 8031 20428 +12397
- Misses 8287 15145 +6858
- Partials 854 1650 +796 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
1b8232c to
2f7821f
Compare
There was a problem hiding this comment.
Pull request overview
Adds a read-through, TTL-based cache to the ZooKeeper Config Center with watch-driven updates, aiming to reduce repeated ZooKeeper reads while keeping cached configuration fresh via existing change events and safer reconnect handling.
Changes:
- Introduces an LRU + TTL config cache with per-path watch state, auto-watch limiting, and generation-based reconnect isolation.
- Updates ZooKeeper config-center listener/watch handling to keep cache entries in sync on add/update/delete events and to avoid duplicate watch registration.
- Adds configuration parsing for
config-center.cache-ttlplus extensive unit tests for cache/watches/reconnect/base64 behavior.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| remoting/zookeeper/listener.go | Adds optional watch-state callbacks and conditional watch re-registration behavior for configuration events. |
| config_center/zookeeper/listener.go | Integrates cache/watch-state tracking into CacheListener, plus reconnect-time watch restoration logic. |
| config_center/zookeeper/listener_test.go | Expands listener tests to cover cache interactions and watch state transitions. |
| config_center/zookeeper/impl.go | Wires in the new cache, adds TTL parsing, updates GetProperties to use read-through cache + watch logic, and resets on reconnect. |
| config_center/zookeeper/impl_test.go | Adds tests for watch selection, watch limits, watch-updated cache reads, base64 decoding from cache, and reconnect/reset behavior. |
| config_center/zookeeper/config_cache.go | Implements the LRU+TTL cache, auto-watch limiting, per-path locking, and generation-based reset behavior. |
| config_center/zookeeper/config_cache_test.go | Adds concurrency, eviction, expiry, reset, and watch-bounding tests for the cache implementation. |
| common/constant/key.go | Adds the config-center.cache-ttl constant. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
we are fixing the ci fail #3669 |
|
|
执行 若要解决这个问题，可以在dubbogo/go-zookeeper中将 |
提一下吧 |
fixed；已在go-zookeeper侧，将invalidateWatcher改为原子读：dubbogo/go-zookeeper#12 |
我发新tag了 go-zookeeper/v1.0.5 你更新到新版本吧 |
更新到1.0.5后，本地跑make test-race，gost编译失败： 原因是更新后的zk将部分测试api搬到了_test.go（WithRetryTimes是直接被删除了），所以这些api不会被编译到普通依赖包，而gost还用的之前的api，编译失败；如果要修这个，又要去gost那边将 |
ok 去 gost 也提一个pr吧 |
gost已经将TestCluster、StartTestCluster、WithRetryTimes转移到本地维护了：dubbogo/gost#145 但在这之后，我尝试dubbogo依赖本地gost，重新跑了一遍make test-race，但etcd服务没连上导致测试超时，测试fail；原因是gost之前移除了grpc.WithBlock()，grpc.DialContext 不等待 Ready直接返回，超时没覆盖到后续Grant；所以我在另一个pr(https://github.com/dubbogo/gost/pull/146)中，在keepSession中，用独立的短超时 context发起首次Grant，且保留了gost删除WithBlock()的行为 |
What this PR does:
This PR adds a read-through cache for the ZooKeeper Config Center to reduce repeated ZooKeeper reads from
GetPropertiesandGetInternalProperty. The existingCacheListenerupdates or invalidates the cache when configuration change events are received.Main changes:
config-center.cache-ttlconfiguration:30s;0disables the cache;Getwhen the watch is active;GetWto read the data and register a data watch when the watch is inactive;ExistsWto watch for node creation when the node does not exist.CacheListenerupdate the cache when add, update, and delete events are received:activestate to track whether the watch for a specific configuration path is still valid, preventing duplicate watch registration during TTL refreshes.AddListenerand `RemoveListener so that the read path, listener path, and listener removal path remain consistent.This PR also adds tests covering cache behavior, TTL, watch-based updates, deletion, reconnect handling, generation isolation, and Base64 reads.
Design options:
This PR evaluated the following three approaches:
TTL-only caching
Advantages:
Disadvantages:
CacheListenerchange notifications.Watch-driven caching only
Advantages:
Disadvantages:
TTL plus watch-driven caching
Advantages:
Getis used when the watch is active, avoiding duplicate watch registration;GetW/ExistsWare used to re-establish the watch when it is inactive;Disadvantages:
The TTL plus watch-driven approach was selected because it satisfies all of the following goals:
activestate design:ZooKeeper watches are one-shot. After a configuration change is received, the existing watch is consumed. The cache therefore needs to know whether a valid watch still exists for each concrete configuration path.
active=true: The path has a valid watch. When the TTL expires, ordinaryGetis used to refresh the content without registering another watch.active=false: The path does not have a valid watch. When the TTL expires,GetWis used to read the content and register a new data watch. If the node does not exist,ExistsWis used to watch for its creation.This state prevents duplicate watch registration during TTL refreshes and ensures that watches are re-established after they are consumed.
Reconnect handling:
After a ZooKeeper reconnect, the cache content and local watch state established before the reconnect can no longer be treated as reliable. The reconnect callback therefore:
The next configuration read after reconnecting accesses ZooKeeper again and re-establishes the watch based on the current node state.
Which issue(s) this PR fixes:
Fixes #3572
Does this PR introduce a user-facing change?:
This PR adds the following user-configurable option: