Skip to content

fix: filter auto-idempotency from cleaner progress checks#12689

Open
rahulkodali wants to merge 1 commit intoapple:mainfrom
rahulkodali:idempotency-test-fix
Open

fix: filter auto-idempotency from cleaner progress checks#12689
rahulkodali wants to merge 1 commit intoapple:mainfrom
rahulkodali:idempotency-test-fix

Conversation

@rahulkodali
Copy link

@rahulkodali rahulkodali commented Feb 10, 2026

Description

Issue #12581

The automatic idempotency test used the oldest remaining idempotency ID as a proxy for cleaner progress. When automatic idempotency is enabled, the client deletes its IDs immediately after commit, which can make the “oldest remaining” jump forward and falsely trigger “cleaned too far.” This change records whether each transaction used automatic idempotency and filters the cleaner progress checks to non‑auto IDs only.

Changes

  • Add automatic flag to the workload’s stored ValueType
  • Record automatic per transaction in _start
  • Filter getCreatedTimes and getOldestCreatedTime to only non‑auto IDs
  • Adjust getOldestCreatedTime to scan idempotency keys until it finds a non‑auto entry

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@rahulkodali rahulkodali marked this pull request as ready for review February 10, 2026 01:02
@Ronitsabhaya75
Copy link
Contributor

@rahulkodali I have some questions here:

  1. The ValueType struct now serializes 3 fields instead of 2. Will this work correctly with existing test data from before this change, or does the test always start fresh?"

  2. Have you tested this with various automaticPercentage values (especially 0%, 50%, and 100%) to ensure the filtering works correctly in all scenarios?

  3. When automaticPercentage=1.0 (100% automatic), getOldestCreatedTime() will return -1. Are all callers prepared to handle this case? I see testCleanerOneIteration uses this value. does it handle -1 correctly?

@rahulkodali
Copy link
Author

@rahulkodali I have some questions here:

  1. The ValueType struct now serializes 3 fields instead of 2. Will this work correctly with existing test data from before this change, or does the test always start fresh?"
  2. Have you tested this with various automaticPercentage values (especially 0%, 50%, and 100%) to ensure the filtering works correctly in all scenarios?
  3. When automaticPercentage=1.0 (100% automatic), getOldestCreatedTime() will return -1. Are all callers prepared to handle this case? I see testCleanerOneIteration uses this value. does it handle -1 correctly?
  1. The simulation test starts from a fresh in‑memory database each run, so there’s no persisted workload data to deserialize across runs. This change is safe for simulation tests. If someone were to reuse a live cluster with old test keys, those old values would deserialize without the new automatic field and be undefined, but that’s not how these simulation workloads are run as far as I know.

  2. Yes, I ran the simulation test with automaticPercentage set to 0%, 50%, and 100%, and it passed in all cases.

  3. Yes, testCleanerOneIteration already treats -1 as “no meaningful progress” and returns true (early exit). That behavior is preserved. In testCleaner, the initial minAgeSeconds is computed from getOldestCreatedTime; with -1 it can go negative, but the loop’s first iteration immediately hits the same -1 in testCleanerOneIteration and exits. This matches the existing semantics (when no IDs remain) prior to these changes.

@rahulkodali rahulkodali reopened this Mar 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants