Skip to content

fix(mito): honor unknown file lingering time#8365

Open
discord9 wants to merge 8 commits into
GreptimeTeam:mainfrom
discord9:omos/unknown-file-lingering-ttl
Open

fix(mito): honor unknown file lingering time#8365
discord9 wants to merge 8 commits into
GreptimeTeam:mainfrom
discord9:omos/unknown-file-lingering-ttl

Conversation

@discord9

Copy link
Copy Markdown
Contributor

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

N/A

What's changed and what's your intention?

This PR restores the intended behavior of region_engine.mito.gc.unknown_file_lingering_time for full-listing GC.

  • Active/open-region unknown files are now deleted only when their object-store last_modified timestamp is older than now - unknown_file_lingering_time.
  • Unknown files in dropped regions remain directly deletable during full-listing GC.
  • Files in manifest or tmp refs remain protected.
  • Known removed files still follow ordinary lingering_time eligibility and are not affected by unknown_file_lingering_time.
  • The default unknown_file_lingering_time is changed from 1h to 1d to reduce risk of deleting pre-manifest files from long-running flush/compaction.
  • Config docs, datanode example config, and the HTTP config snapshot are updated.
  • GC worker tests now cover the active unknown TTL logic, dropped-region behavior, known removed-file behavior, and fast mode not processing unknown files.

Verification performed:

  • cargo test -p mito2 -- "gc::worker_test" --nocapture
  • cargo check -p mito2 --tests
  • git diff --check

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the garbage collection (GC) logic for Mito engine files. Specifically, it changes the default unknown_file_lingering_time from 1 hour to 1 day to prevent accidental deletion of pre-manifest files, and updates should_delete_file to check the object's last_modified timestamp against this TTL for active regions. It also refactors the GC worker code to extract list_to_be_deleted_files_impl and filter_deletable_files as standalone functions, accompanied by comprehensive unit tests. The review feedback suggests making list_to_be_deleted_files_impl synchronous, as it contains no asynchronous operations, which would eliminate unnecessary future-polling overhead in both the production code and the newly added tests.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/mito2/src/gc.rs Outdated
Comment thread src/mito2/src/gc.rs Outdated
Comment thread src/mito2/src/gc/worker_test.rs Outdated
discord9 added 4 commits June 30, 2026 11:31
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
@discord9 discord9 force-pushed the omos/unknown-file-lingering-ttl branch from 275b984 to f28591b Compare June 30, 2026 03:33
@discord9 discord9 marked this pull request as ready for review July 1, 2026 06:49
@discord9 discord9 requested review from a team, evenyag, v0y4g3r and waynexia as code owners July 1, 2026 06:49
Signed-off-by: discord9 <discord9@163.com>
discord9 added 3 commits July 2, 2026 11:20
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-required This change requires docs update. size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants