Skip to content

executor: skip analyze flush broadcast in test binaries#68417

Merged
ti-chi-bot[bot] merged 1 commit into
pingcap:masterfrom
0xPoe:poe-patch-bench
May 18, 2026
Merged

executor: skip analyze flush broadcast in test binaries#68417
ti-chi-bot[bot] merged 1 commit into
pingcap:masterfrom
0xPoe:poe-patch-bench

Conversation

@0xPoe
Copy link
Copy Markdown
Member

@0xPoe 0xPoe commented May 15, 2026

What problem does this PR solve?

Issue Number: close #68416

Problem Summary:
make bench-daily (CI job gobench4) fails on BenchmarkNonPartitionPointGetPlanCacheOn: analyze table t hits context deadline exceeded after 10s. PR #67939 added flushStatsDeltaForAnalyze, which broadcasts FLUSH STATS_DELTA CLUSTER via a TiDB-type cop request before each ANALYZE. The local-dump fallback was gated on intest.InTest, but that variable is only set under the intest build tag. make bench-daily runs go test without the tag, so the broadcast hits the mockstore session's empty :10080 and hangs.

What changed and how does it work?

Replace the gate with a runningUnderGoTest helper backed by testing.Testing(). The Go linker sets testing.Testing() to true for any binary produced by go test (covering both unit tests and make bench-daily) and leaves it false for go build of cmd/tidb-server. The gate is therefore a no-op in production regardless of host or flag configuration, and no longer depends on the intest build tag.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Reproduction and verification (same machine, same command):

#68417 (comment)

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

Summary by CodeRabbit

  • Chores
    • Improved test execution efficiency by optimizing internal statistics handling during testing. Test processes now use a more efficient local flushing approach instead of cluster-level broadcasts, reducing unnecessary background operations during test runs.

Review Change Stack

@ti-chi-bot ti-chi-bot Bot added the release-note-none Denotes a PR that doesn't merit a release note. label May 15, 2026
@pantheon-ai
Copy link
Copy Markdown

pantheon-ai Bot commented May 15, 2026

@0xPoe I've received your pull request and will start the review. I'll conduct a thorough review covering code quality, potential issues, and implementation details.

⏳ This process typically takes 10-30 minutes depending on the complexity of the changes.

ℹ️ Learn more details on Pantheon AI.

@ti-chi-bot ti-chi-bot Bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label May 15, 2026
@0xPoe 0xPoe changed the title executor: skip analyze flush broadcast when AdvertiseAddress is empty WIP: executor: skip analyze flush broadcast when AdvertiseAddress is empty May 15, 2026
@ti-chi-bot ti-chi-bot Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 15, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 9afd1f7e-bb6f-48c7-9b77-d9e0cd875c12

📥 Commits

Reviewing files that changed from the base of the PR and between dcea072 and 46c8ff8.

📒 Files selected for processing (1)
  • pkg/executor/analyze.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/executor/analyze.go

📝 Walkthrough

Walkthrough

Replace the intest.InTest build-tag gate with a runtime runningUnderGoTest() helper (uses testing.Testing()) so flushStatsDeltaForAnalyze flushes stats deltas locally when running under Go test binaries.

Changes

Analyze stats delta flushing topology detection

Layer / File(s) Summary
In-process test detection with testing.Testing()
pkg/executor/analyze.go
Add testing package import, introduce runningUnderGoTest() helper that returns testing.Testing() and documents in-process test domains lacking a real TiDB RPC listener, and update flushStatsDeltaForAnalyze to use this helper instead of intest.InTest.

Sequence Diagram

sequenceDiagram
  participant Analyze
  participant runningUnderGoTest
  participant FlushAction
  Analyze->>runningUnderGoTest: call testing.Testing()
  runningUnderGoTest-->>Analyze: bool result
  Analyze->>FlushAction: [true] local dump
  Analyze->>FlushAction: [false] cluster FLUSH STATS_DELTA
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • pingcap/tidb#67939: Introduced the flushStatsDeltaForAnalyze function with the original intest.InTest gate; this PR fixes its topology detection for go test execution without build tags.

Suggested labels

sig/planner, component/statistics, ok-to-test

Suggested reviewers

  • henrybw
  • qw4990
  • terry1purcell

Poem

🐰 I sniff the test-time breeze,
No build tag hides the tease,
testing.Testing() lights the way,
Local flush keeps broadcasts at bay,
Hooray — no mockstore delays! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'executor: skip analyze flush broadcast in test binaries' clearly and concisely describes the main change: modifying executor behavior to skip the broadcast in test binaries.
Description check ✅ Passed The PR description includes the required issue reference (close #68416), explains the problem and solution, and documents manual test verification.
Linked Issues check ✅ Passed The changes fully address issue #68416 by replacing the intest.InTest gate with runningUnderGoTest() backed by testing.Testing(), which correctly detects Go test binaries and prevents broadcast hangs.
Out of Scope Changes check ✅ Passed All changes are within scope of fixing the benchmark timeout by improving test detection for conditional stats-delta flushing; no unrelated modifications are present.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 75.6786%. Comparing base (8c17ce1) to head (46c8ff8).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #68417        +/-   ##
================================================
- Coverage   77.2761%   75.6786%   -1.5975%     
================================================
  Files          2010       2008         -2     
  Lines        555473     564521      +9048     
================================================
- Hits         429248     427222      -2026     
- Misses       125305     137195     +11890     
+ Partials        920        104       -816     
Flag Coverage Δ
integration 41.1463% <66.6666%> (+1.3522%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4679% <ø> (ø)
parser ∅ <ø> (∅)
br 49.9725% <ø> (-13.0354%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@0xPoe 0xPoe force-pushed the poe-patch-bench branch from 7510a45 to 0c2cbfd Compare May 18, 2026 09:31
@ti-chi-bot ti-chi-bot Bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 18, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/executor/analyze_utils_test.go (1)

74-81: ⚡ Quick win

Avoid replacing the entire global config in this test.

Using config.NewConfig() and StoreGlobalConfig(...) here swaps all global config fields, which increases cross-test coupling risk. Prefer preserving the existing config and changing only AdvertiseAddress for each case.

As per coding guidelines, keep test changes minimal and deterministic; avoid broad churn unless required.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/executor/analyze_utils_test.go` around lines 74 - 81, The test currently
replaces the entire global config using config.NewConfig() +
config.StoreGlobalConfig(...), which is risky; instead, read the existing global
config, copy it and only change the AdvertiseAddress field for the two cases,
call config.StoreGlobalConfig with that modified copy, and ensure you restore
the original config with defer after the test; update the instances where
AdvertiseAddress is set (referencing AdvertiseAddress, config.StoreGlobalConfig,
and isInProcessTopology) so only AdvertiseAddress is mutated and the rest of the
global config remains unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/executor/analyze_utils_test.go`:
- Around line 74-81: The test currently replaces the entire global config using
config.NewConfig() + config.StoreGlobalConfig(...), which is risky; instead,
read the existing global config, copy it and only change the AdvertiseAddress
field for the two cases, call config.StoreGlobalConfig with that modified copy,
and ensure you restore the original config with defer after the test; update the
instances where AdvertiseAddress is set (referencing AdvertiseAddress,
config.StoreGlobalConfig, and isInProcessTopology) so only AdvertiseAddress is
mutated and the rest of the global config remains unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 2a92f652-e0af-42b3-a403-759f857dd18b

📥 Commits

Reviewing files that changed from the base of the PR and between 7510a45 and 0c2cbfd.

📒 Files selected for processing (3)
  • pkg/executor/BUILD.bazel
  • pkg/executor/analyze.go
  • pkg/executor/analyze_utils_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/executor/analyze.go

@0xPoe 0xPoe force-pushed the poe-patch-bench branch from 0c2cbfd to fc0d2ad Compare May 18, 2026 10:00
@ti-chi-bot ti-chi-bot Bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 18, 2026
@0xPoe 0xPoe force-pushed the poe-patch-bench branch from fc0d2ad to dcea072 Compare May 18, 2026 10:08
@0xPoe
Copy link
Copy Markdown
Member Author

0xPoe commented May 18, 2026

Tested locally:

go test -run '^$' -bench '^BenchmarkNonPartitionPointGetPlanCacheOn$' \
        -benchtime 1x -timeout 60s ./pkg/planner/core/tests/partition

Before

make bench-daily builds without the intest tag, so the gate stays closed, the broadcast targets the mockstore session's empty :10080, and the analyze setup hits the 10s context deadline:

[FATAL] mustExecute error sql="analyze table t" error="context deadline exceeded"
  flushStatsDeltaForAnalyze   pkg/executor/analyze.go:128
  broadcast                   pkg/executor/simple.go:3034
  (*copIterator).Next         pkg/store/copr/coprocessor.go:1445
  preparePointGet             pkg/planner/core/tests/partition/bench_test.go:200
  BenchmarkNonPartitionPointGetPlanCacheOn   pkg/planner/core/tests/partition/bench_test.go:313

exit status 1
FAIL    github.com/pingcap/tidb/pkg/planner/core/tests/partition    12.414s

After

BenchmarkNonPartitionPointGetPlanCacheOn-14        1    202458 ns/op
PASS
ok  github.com/pingcap/tidb/pkg/planner/core/tests/partition    2.333s

@0xPoe 0xPoe changed the title WIP: executor: skip analyze flush broadcast when AdvertiseAddress is empty WIP: executor: skip analyze flush broadcast in test binaries May 18, 2026
@0xPoe 0xPoe changed the title WIP: executor: skip analyze flush broadcast in test binaries executor: skip analyze flush broadcast in test binaries May 18, 2026
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 18, 2026
@0xPoe 0xPoe force-pushed the poe-patch-bench branch from dcea072 to ee183d7 Compare May 18, 2026 10:22
`make bench-daily` (CI job `gobench4`) fails on
`BenchmarkNonPartitionPointGetPlanCacheOn`: `analyze table t` hits
`context deadline exceeded` after 10s. PR pingcap#67939 added
`flushStatsDeltaForAnalyze`, which broadcasts FLUSH STATS_DELTA CLUSTER
via a TiDB-type cop request before each ANALYZE. The local-dump
fallback was gated on `intest.InTest`, but that variable is only set
under the `intest` build tag. `make bench-daily` runs `go test`
without the tag, so the broadcast hits the mockstore session's empty
`:10080` and hangs.

Replace the gate with a `runningUnderGoTest` helper backed by
`testing.Testing()`, which the linker sets to true for any binary
produced by `go test` (covering both unit tests and bench-daily) and
leaves false for `go build` of cmd/tidb-server. The gate is therefore
a no-op in production regardless of TOML host configuration, and no
longer depends on the `intest` build tag.

Issue Number: close pingcap#68416
@0xPoe 0xPoe force-pushed the poe-patch-bench branch from ee183d7 to 46c8ff8 Compare May 18, 2026 10:23
Copy link
Copy Markdown
Member Author

@0xPoe 0xPoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check (PR reviewed by myself and ready for feedback)

  • Code compiles successfully

  • Unit tests added

  • No AI-generated elegant nonsense in PR.

  • Comments added where necessary

  • PR title and description updated

  • Documentation PR created (or confirmed not needed)

  • PR size is reasonable

/cc @terry1purcell @mjonss

@ti-chi-bot ti-chi-bot Bot requested review from mjonss and terry1purcell May 18, 2026 10:28
Comment thread pkg/executor/analyze.go
Copy link
Copy Markdown
Contributor

@mjonss mjonss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot Bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels May 18, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented May 18, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mjonss, terry1purcell

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels May 18, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented May 18, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-05-18 12:14:24.789139781 +0000 UTC m=+179994.293270457: ☑️ agreed by mjonss.
  • 2026-05-18 14:39:43.358006949 +0000 UTC m=+188712.862137625: ☑️ agreed by terry1purcell.

@0xPoe
Copy link
Copy Markdown
Member Author

0xPoe commented May 18, 2026

/retest

@ti-chi-bot ti-chi-bot Bot merged commit 2ce45f0 into pingcap:master May 18, 2026
36 checks passed
@0xPoe 0xPoe added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label May 18, 2026
@ti-chi-bot
Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #68471.

@0xPoe
Copy link
Copy Markdown
Member Author

0xPoe commented May 18, 2026

Thanks for your review! 💚 💙 💜 💛 ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make bench-daily fails: BenchmarkNonPartitionPointGetPlanCacheOn times out on analyze table t

4 participants