Skip to content

Commit 1c416b3

Browse files
authored
ci: warm test-bench cache on push to main (#19968)
## Problem `test-bench` only triggered on `pull_request`, so there was never a base-branch cache on `main`. GitHub's cache isolation allows workflow runs to fall back to the base branch — but not to other PRs or branches. Without a `main` cache: - Every new PR's first push starts completely cold (full module download + recompile) - Merge-queue runs (`merge_group`) also start cold — they can read from `main` but not from other PRs - Caches saved during merge-group runs are stored under the temporary `gh-readonly-queue/...` branch and become inaccessible once the merge group completes; they never contribute to `main` ## Fix Add `push` triggers for `main` and `release/**`. After each merge, a cache-warming run compiles all test binaries and saves them under the `main` branch scope. Future PR first-pushes and merge-queue runs restore from that warm cache. On non-`pull_request` events, the benchmark execution step is replaced with `go test -run=^$ ./...` — this compiles every test binary (producing identical GOCACHE entries) but runs nothing. The job always succeeds on `main` because correctness is already guaranteed by the merge queue before the commit lands. Also adds `workflow_dispatch` (required by project guidelines) and documents the cache isolation model and compile-only warming pattern in `.github/README.md`.
1 parent ddcba72 commit 1c416b3

2 files changed

Lines changed: 72 additions & 5 deletions

File tree

.github/README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,58 @@ unnecessary re-runs.
9494
This is also why the `execution/tests/` package is broken into focused sub-packages
9595
rather than one monolithic package.
9696

97+
### Cache warming and GitHub's cache isolation rules
98+
99+
GitHub Actions caches are scoped by branch. A workflow run can restore caches from:
100+
101+
1. **Its own branch** — caches saved by earlier runs on the same branch.
102+
2. **The base branch** (`main`) — caches saved by any workflow run on `main`.
103+
104+
It cannot access caches from other PRs or other branches.
105+
106+
**The cold-start problem** — If a workflow only triggers on `pull_request`, it never
107+
runs on `main` and therefore never populates the base-branch cache. Every new PR
108+
opens to a cold cache, even if the codebase hasn't changed at all. Subsequent pushes
109+
to the *same* PR do find each other's caches, but only until the GitHub-hosted cache
110+
is evicted (repos share a 10 GB limit; busy repos evict older entries within hours).
111+
112+
**The fix** — add `push: {branches: [main, release/**]}` as a trigger. After each
113+
merge, a run warms the build and module caches on `main`. All future PR first-pushes
114+
restore from that warm base-branch cache instead of starting cold.
115+
116+
**Cache warming without test validation** — For workflows where correctness is already
117+
enforced by the merge queue, the push-to-main run exists purely for cache warming. To
118+
prevent benchmark or test flakiness from showing as failures on main commits, use
119+
compile-only steps on non-PR events:
120+
121+
```yaml
122+
- name: Run benchmarks
123+
if: github.event_name == 'pull_request'
124+
run: make test-bench
125+
126+
- name: Build test binaries (cache warming)
127+
if: github.event_name != 'pull_request'
128+
run: go test -run=^$ ./...
129+
```
130+
131+
`go test -run=^$` compiles every test binary (identical GOCACHE entries to a full
132+
run) but executes nothing. The job always succeeds, and the commit on `main` shows
133+
green. If a benchmark were broken post-merge it would have been caught by the PR or
134+
merge queue run before landing.
135+
136+
**Merge queues** — `merge_group` runs use a temporary synthetic branch
137+
(`refs/heads/gh-readonly-queue/main/pr-N-SHA`). They can read from the base-branch
138+
(`main`) cache, so they benefit from the warm cache created by the `push` trigger.
139+
However, caches *saved* during a merge-group run are stored under that temporary
140+
branch and become inaccessible once the merge group completes — they do not
141+
contribute to the `main` cache. Only the `push`-triggered run after the merge creates
142+
a durable base-branch cache.
143+
144+
**Implication for workflow design**: any workflow that should benefit from caching
145+
across PRs and merge queue runs needs a `push` trigger on `main` (or a nightly
146+
schedule on `main`). Without it, each PR and each merge-queue batch is always a cold
147+
start.
148+
97149
## Required checks
98150

99151
Required checks exist to block *regressions*, not to enforce perfection.

.github/workflows/test-bench.yml

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
11
name: Benchmarks
22

3-
# We only run benchmarks on pull requests to catch accidental breakage early.
4-
# We don't run on push to main/release because benchmarks aren't actively used
5-
# for anything right now — if they break post-merge it's not a release blocker.
6-
# If we ever care about benchmark results over time, that would be a separate
7-
# workflow on main that tracks performance trends, not this one.
3+
# Run benchmarks on pull requests to catch accidental breakage, and on push
4+
# to main/release to keep a warm build cache. Without a push trigger,
5+
# test-bench never runs on main and every new PR starts from a cold cache,
6+
# because GitHub's cache fallback chain only reaches the base branch.
87
on:
98
pull_request:
109
branches:
@@ -14,6 +13,17 @@ on:
1413
- reopened
1514
- synchronize
1615
- ready_for_review
16+
push:
17+
# Cache warming only — benchmarks are not re-run on push to main/release.
18+
# GitHub's cache isolation means PR and merge-queue runs can only fall back
19+
# to the base-branch cache. Without this trigger, test-bench never runs on
20+
# main so every new PR starts cold. The push run compiles test binaries
21+
# (warming GOCACHE and GOMODCACHE) but does not execute them; correctness
22+
# is already guaranteed by the merge queue before the commit lands here.
23+
branches:
24+
- main
25+
- release/**
26+
workflow_dispatch:
1727

1828
defaults:
1929
run:
@@ -40,7 +50,12 @@ jobs:
4050
id: erigon
4151

4252
- name: Run benchmarks
53+
if: github.event_name == 'pull_request'
4354
run: make test-bench
4455

56+
- name: Build test binaries (cache warming)
57+
if: github.event_name != 'pull_request'
58+
run: go test -run=^$ ./...
59+
4560
- uses: ./.github/actions/cleanup-erigon
4661
if: always()

0 commit comments

Comments
 (0)