[manager] move EventReport cleanup into background GC - #281
Conversation
shaohuaxi
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8770c29720
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
![qoderai[bot]](https://avatars.githubusercontent.com/in/1396158?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Summary
Reviewed PR #281 (EventReport active cleanup integrated into background scan GC). The design document (docs/design/event_report_background_gc.md) is thorough and well-structured. The intent model, three-state lifecycle lease, per-instance cursor pass, fair scheduling, and maintenance metadata API contracts are clearly documented and faithfully implemented.
Key strengths
- Intent merge/replace semantics correctly handle event storms (multiple Snapshots/HOST_DOWN collapse into per-Instance passes).
- Maintenance no-touch reads (
ApplyToEntryNoTouch,GetForOneKeyForMaintenance) prevent LRU/access-time pollution — the two-phaseScanLocationsForMaintenancerefactor inmeta_local_backend.ccis a clean fix for the prior shard-mutex-during-item-read concern. - Accounting safety: prior mutation flags are carried across Sync failures (
meta_searcher.cclines 2001-2007) to prevent double-counting, andCommitMaintenanceDeleteAccountingonly runs after all hard errors converge. - Shutdown ordering (RequestStop → ClearCallbacks → Join → Reset) and fail-closed restart semantics (no persisted generation, intent/cursor discarded) are sound.
- Test coverage is comprehensive across intent lifecycle, lease states, maintenance delete, metrics, and E2E.
Findings (3 inline comments)
- kBusy blocks entire batch (
cache_garbage_collector.cc:1263): One busy reporter fails the whole action batch, preventing progress for unrelated reporters. Considercontinue+out_revisit_required=true. - kStale early return skips remaining targets (
cache_garbage_collector.cc:1275): Inconsistent withBeginEventReportPass(line 973 usescontinue). Remaining targets are deferred to a full pass rescan unnecessarily. - String-based metric lookups on hot path (
cache_garbage_collector.cc:1506): Event report metrics useGetCounter("cache_gc.event_report_*")per call, unlike the regular GC path which uses cachedMETRICS_macros.
All findings are low-to-medium severity — the implementation is functionally correct and the issues are about efficiency and consistency.
🤖 Generated by Qoder
e2d6708 to
8f48e72
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f48e72957
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
5a0246e to
0939176
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 09391763ac
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
9a20dbb to
67cfab2
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 67cfab2844
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
67cfab2 to
879020a
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 879020af84
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
879020a to
50fcbee
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 50fcbeecd7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
50fcbee to
565e0ad
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 565e0ad5b0
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
d46cd5e to
403a294
Compare
wangxiyu191
left a comment
wangxiyu191
left a comment
There was a problem hiding this comment.
建议把这部分从事件触发清理改成 GC 按固定频率持续扫描。
Snapshot、HOST_DOWN 等事件只更新 DataStorage 的当前状态，不再向 GC 登记清理任务。GC 扫描到 Location 时，找到对应的 DataStorage，由 DataStorage 判断当前是否需要删除；暂时无法判断就跳过，下一轮再检查。EventReport 和普通 GC 可以在同一轮扫描里处理，不需要分别维护两套扫描流程。
这样可以删掉 Intent map 和 EventReport 专用的调度、扫描状态，整体逻辑会简单很多。后续 DataStorage 的 MightExist 定期检查、TTL 到期回收等需求，也可以直接复用这套扫描流程，只增加各自的判断逻辑，不需要每增加一种清理原因就再做一套事件触发和全表扫描。
403a294 to
a56b7cc
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a56b7cc7a3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
8593875 to
bf1cd1d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf1cd1d7e4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
bf1cd1d to
be0f937
Compare
2 participants
Background
EventReport currently schedules an instance-wide metadata scan after every successful Snapshot and HOST_DOWN/liveness cleanup. Repeated events can therefore multiply full scans and compete with deletion and migration work in the shared executor.
This PR builds on #244 and moves those cleanup paths into the resident background GC. EventReport events register or advance cleanup intents; GC merges compatible intents and reconciles them with bounded per-instance scans.
Changes
Correctness boundaries
Dependency
This is the second layer of a stacked change and depends on #244.
Stack created with GitHub Stacks CLI • Give Feedback 💬