Skip to content

util: aggregate write RUv2 into commit details#1939

Merged
ti-chi-bot[bot] merged 4 commits intotikv:masterfrom
oh-my-tidb:disksing/ruv2-commit-details
Apr 9, 2026
Merged

util: aggregate write RUv2 into commit details#1939
ti-chi-bot[bot] merged 4 commits intotikv:masterfrom
oh-my-tidb:disksing/ruv2-commit-details

Conversation

@disksing
Copy link
Copy Markdown
Collaborator

@disksing disksing commented Apr 8, 2026

Summary

  • aggregate write-path RUv2 into CommitDetails
  • store only aggregated RUV2 instead of the whole ExecDetailsV2
  • expose a GetWriteRUV2() accessor for TiDB-side statement metrics aggregation

Tests

  • go test ./util -run 'TestCommitDetails|TestLockKeysDetails' -count=1

Summary by CodeRabbit

  • Chores

    • Improved tracking and aggregation of per-executor write-operation metrics and ensured safe deep-copying of aggregated metrics.
    • Updated integration test module dependencies and replacements to align with the current test harness.
  • Tests

    • Added comprehensive tests validating execution-details aggregation, deep-copy isolation, and aggregated metric increments.

Signed-off-by: disksing <i@disksing.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. labels Apr 8, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 8, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e670fefc-ddcd-47d3-87fc-70e6d8864878

📥 Commits

Reviewing files that changed from the base of the PR and between 1a0c74a and 79feda1.

⛔ Files ignored due to path filters (1)
  • integration_tests/go.sum is excluded by !**/*.sum
📒 Files selected for processing (1)
  • integration_tests/go.mod
🚧 Files skipped from review as they are similar to previous changes (1)
  • integration_tests/go.mod

📝 Walkthrough

Walkthrough

Added safe cloning and aggregation of kvrpcpb.RUV2 into CommitDetails via a new WriteRUV2 field, updated merge paths to accumulate RUv2 from prewrite/commit exec details and increment write RPC counts, and exposed GetWriteRUV2() for retrieving a cloned aggregate.

Changes

Cohort / File(s) Summary
RUV2 Aggregation & CommitDetails Extension
util/execdetails.go
Added WriteRUV2 *kvrpcpb.RUV2 to CommitDetails.Mu. Implemented cloneRUV2, mergeCommitDetailsRUV2, mergeRUV2. Updated CommitDetails.Merge, MergePrewriteReqDetails, and MergeCommitReqDetails to aggregate RUv2 and increment WriteRpcCount. Added GetWriteRUV2() and deep-copy support in Clone().
Tests for RUV2 Behavior
util/execdetails_test.go
Added test helpers newWriteRUV2, newWriteExecDetailsV2. Extended TestCommitDetailsMerge and TestCommitDetailsClone to validate WriteRUV2 fields and deep-copy semantics. Added TestCommitDetailsMergeWriteRUV2 to assert correct aggregation from prewrite/commit merges and immutability of returned clones.
Integration Test Dependencies
integration_tests/go.mod
Updated module dependency versions and added replace directives pointing to alternative TiDB module paths. (Dependency changes only; no code logic edits.)

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

lgtm, approved

Suggested reviewers

  • cfzjywxk
  • JmPotato
  • ekexium

Poem

🐰 I nibble bytes and metrics bright,

I clone each RUV2 through the night,
I stitch prewrite and commit in tune,
Counters hum beneath the moon,
A rabbit cheers — aggregation's delight.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: aggregating write RUv2 metrics into the CommitDetails struct for tracking execution details.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ti-chi-bot ti-chi-bot Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 8, 2026
@disksing disksing marked this pull request as ready for review April 8, 2026 15:07
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 8, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
util/execdetails.go (1)

249-275: ⚠️ Potential issue | 🟡 Minor

PrewriteReqNum is still dropped in Merge and Clone.

This change preserves WriteRUV2, but PrewriteReqNum is still neither accumulated nor copied. Any merged or cloned CommitDetails will silently lose that counter.

🐛 Proposed fix
 func (cd *CommitDetails) Merge(other *CommitDetails) {
 	cd.ResolveLock.ResolveLockTime += other.ResolveLock.ResolveLockTime
 	cd.WriteKeys += other.WriteKeys
 	cd.WriteSize += other.WriteSize
 	cd.PrewriteRegionNum += other.PrewriteRegionNum
+	cd.PrewriteReqNum += other.PrewriteReqNum
 	cd.TxnRetry += other.TxnRetry
 	cd.Mu.CommitBackoffTime += other.Mu.CommitBackoffTime
@@
 	commit := &CommitDetails{
 		GetCommitTsTime:        cd.GetCommitTsTime,
 		GetLatestTsTime:        cd.GetLatestTsTime,
 		LagDetails:             cd.LagDetails,
 		PrewriteTime:           cd.PrewriteTime,
@@
 		WriteKeys:              cd.WriteKeys,
 		WriteSize:              cd.WriteSize,
 		PrewriteRegionNum:      cd.PrewriteRegionNum,
+		PrewriteReqNum:         cd.PrewriteReqNum,
 		TxnRetry:               cd.TxnRetry,
 		ResolveLock:            cd.ResolveLock,
 	}

Also applies to: 331-353

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@util/execdetails.go` around lines 249 - 275, The Merge implementation for
CommitDetails is missing aggregation of the PrewriteReqNum counter and the Clone
method likewise doesn't copy it, causing that metric to be lost; update
CommitDetails.Merge to add cd.PrewriteReqNum += other.PrewriteReqNum (alongside
the other += accumulations) and update CommitDetails.Clone to set the cloned
object's PrewriteReqNum = original.PrewriteReqNum so clones preserve the value
(refer to the methods named CommitDetails.Merge and CommitDetails.Clone and the
field PrewriteReqNum).
🧹 Nitpick comments (1)
util/execdetails_test.go (1)

294-307: Also mutate ExecutorInputs in the copy-isolation checks.

cloneRUV2()'s tricky part is the nested ExecutorInputs pointer, but these assertions only modify top-level scalars. An aliasing regression there would still pass.

🧪 Suggested test additions
 	cloned.Mu.WriteRUV2.KvEngineCacheMiss = 999
 	assert.Equal(t, uint64(1), orig.Mu.WriteRUV2.KvEngineCacheMiss)
+	cloned.Mu.WriteRUV2.ExecutorInputs.TikvCoprocessorExecutorWorkTotalBatchIndexScan = 999
+	assert.Equal(t, uint64(6), orig.Mu.WriteRUV2.ExecutorInputs.TikvCoprocessorExecutorWorkTotalBatchIndexScan)
@@
 	agg.KvEngineCacheMiss = 999
 	assert.Equal(t, uint64(111), commitDetails.GetWriteRUV2().KvEngineCacheMiss)
+	agg.ExecutorInputs.TikvCoprocessorExecutorWorkTotalBatchIndexScan = 999
+	assert.Equal(t, uint64(444), commitDetails.GetWriteRUV2().ExecutorInputs.TikvCoprocessorExecutorWorkTotalBatchIndexScan)

Also applies to: 318-332

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@util/execdetails_test.go` around lines 294 - 307, The test misses verifying
deep-copy isolation for the nested ExecutorInputs pointer in cloneRUV2; update
the copy-isolation checks to mutate cloned.Mu.WriteRUV2.ExecutorInputs (e.g.,
change a field or append/remove an entry) and then assert
orig.Mu.WriteRUV2.ExecutorInputs remains unchanged, and apply the same
additional assertion in the second test block (the similar checks around lines
318-332) so the tests will catch aliasing of the nested ExecutorInputs pointer.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@util/execdetails.go`:
- Around line 249-275: The Merge implementation for CommitDetails is missing
aggregation of the PrewriteReqNum counter and the Clone method likewise doesn't
copy it, causing that metric to be lost; update CommitDetails.Merge to add
cd.PrewriteReqNum += other.PrewriteReqNum (alongside the other += accumulations)
and update CommitDetails.Clone to set the cloned object's PrewriteReqNum =
original.PrewriteReqNum so clones preserve the value (refer to the methods named
CommitDetails.Merge and CommitDetails.Clone and the field PrewriteReqNum).

---

Nitpick comments:
In `@util/execdetails_test.go`:
- Around line 294-307: The test misses verifying deep-copy isolation for the
nested ExecutorInputs pointer in cloneRUV2; update the copy-isolation checks to
mutate cloned.Mu.WriteRUV2.ExecutorInputs (e.g., change a field or append/remove
an entry) and then assert orig.Mu.WriteRUV2.ExecutorInputs remains unchanged,
and apply the same additional assertion in the second test block (the similar
checks around lines 318-332) so the tests will catch aliasing of the nested
ExecutorInputs pointer.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e4f7ed72-16d5-4c75-b87a-19ae81479152

📥 Commits

Reviewing files that changed from the base of the PR and between a888f42 and f67451f.

📒 Files selected for processing (2)
  • util/execdetails.go
  • util/execdetails_test.go

Signed-off-by: disksing <i@disksing.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@integration_tests/go.mod`:
- Around line 180-181: Add a short explanatory comment near the replace
directives in integration_tests/go.mod (the lines redirecting
github.com/pingcap/tidb and github.com/pingcap/tidb/pkg/parser to
github.com/oh-my-tidb/tidb) stating that these replaces are temporary for
testing the commit-details feature against the oh-my-tidb fork, and note that
they should be removed once upstream TiDB includes the commit-details changes;
alternatively add the same note to the integration test setup README if you
prefer docs over inline comments.
- Line 20: Update the grpc module version in go.mod from "google.golang.org/grpc
v1.75.1" to "google.golang.org/grpc v1.79.3" (or later) to remediate the
GHSA-p77j-4mvh-x3m3 authorization bypass; after changing the module line, run
"go mod tidy" (and optionally "go test ./...") to refresh the dependency graph
and ensure no other modules pin the older grpc version.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: adafe0a0-04b0-4e26-9c91-2e194003e093

📥 Commits

Reviewing files that changed from the base of the PR and between f67451f and 1a0c74a.

⛔ Files ignored due to path filters (1)
  • integration_tests/go.sum is excluded by !**/*.sum
📒 Files selected for processing (1)
  • integration_tests/go.mod

Comment thread integration_tests/go.mod Outdated
Comment thread integration_tests/go.mod
disksing added 2 commits April 9, 2026 11:24
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 9, 2026
@ti-chi-bot ti-chi-bot Bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Apr 9, 2026
@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 9, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Apr 9, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ekexium, nolouch

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Apr 9, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Apr 9, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-04-09 04:44:43.505805053 +0000 UTC m=+1017888.711165110: ☑️ agreed by nolouch.
  • 2026-04-09 05:22:33.61721142 +0000 UTC m=+1020158.822571467: ☑️ agreed by ekexium.

@ti-chi-bot ti-chi-bot Bot merged commit d1ffcbb into tikv:master Apr 9, 2026
13 checks passed
disksing added a commit to oh-my-tidb/client-go that referenced this pull request Apr 10, 2026
This reverts commit d1ffcbb.

Signed-off-by: disksing <i@disksing.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants