ci: parallelize the repeated test job; add concurrency cancellation#4309
ci: parallelize the repeated test job; add concurrency cancellation#4309ReneWerner87 wants to merge 1 commit into
Conversation
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (2)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR optimizes CI wall-clock time for Fiber’s test and benchmark workflows while keeping coverage upload behavior and existing validation intent intact.
Changes:
- Add PR-only workflow concurrency cancellation to avoid wasted CI on superseded commits.
- Pin
gotestsumtov1.13.0and reduce unnecessary coverage instrumentation to the single job that uploads coverage. - Speed up benchmark runs by reducing
go testbenchtime.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| .github/workflows/test.yml | Cancels superseded PR runs, pins gotestsum, computes coverage only on the upload job, and removes -race from the repeated/flakiness job. |
| .github/workflows/benchmark.yml | Cancels superseded PR runs and reduces benchmark runtime via -benchtime=100ms. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4309 +/- ##
=======================================
Coverage 91.33% 91.33%
=======================================
Files 132 132
Lines 13089 13089
=======================================
Hits 11955 11955
Misses 716 716
Partials 418 418
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.
| Benchmark suite | Current: 5fad032 | Previous: 86397b0 | Ratio |
|---|---|---|---|
Benchmark_NewError (github.com/gofiber/fiber/v3) |
85.77 ns/op 24 B/op 1 allocs/op |
33.9 ns/op 24 B/op 1 allocs/op |
2.53 |
Benchmark_NewError (github.com/gofiber/fiber/v3) - ns/op |
85.77 ns/op |
33.9 ns/op |
2.53 |
Benchmark_Ctx_AutoFormat_JSON (github.com/gofiber/fiber/v3) |
428.9 ns/op 16 B/op 1 allocs/op |
285.3 ns/op 16 B/op 1 allocs/op |
1.50 |
Benchmark_Ctx_AutoFormat_JSON (github.com/gofiber/fiber/v3) - ns/op |
428.9 ns/op |
285.3 ns/op |
1.50 |
Benchmark_Ctx_Write (github.com/gofiber/fiber/v3) |
19.14 ns/op 70 B/op 0 allocs/op |
12.74 ns/op 76 B/op 0 allocs/op |
1.50 |
Benchmark_Ctx_Write (github.com/gofiber/fiber/v3) - ns/op |
19.14 ns/op |
12.74 ns/op |
1.50 |
Benchmark_App_RebuildTree (github.com/gofiber/fiber/v3) |
25290 ns/op 15960 B/op 48 allocs/op |
16340 ns/op 15960 B/op 48 allocs/op |
1.55 |
Benchmark_App_RebuildTree (github.com/gofiber/fiber/v3) - ns/op |
25290 ns/op |
16340 ns/op |
1.55 |
Benchmark_Router_WithCompression (github.com/gofiber/fiber/v3) |
320.6 ns/op 8 B/op 1 allocs/op |
209.5 ns/op 8 B/op 1 allocs/op |
1.53 |
Benchmark_Router_WithCompression (github.com/gofiber/fiber/v3) - ns/op |
320.6 ns/op |
209.5 ns/op |
1.53 |
Benchmark_PaginateMiddlewareWithCustomConfig (github.com/gofiber/fiber/v3/middleware/paginate) |
21657 ns/op 11492 B/op 43 allocs/op |
14378 ns/op 11474 B/op 43 allocs/op |
1.51 |
Benchmark_PaginateMiddlewareWithCustomConfig (github.com/gofiber/fiber/v3/middleware/paginate) - ns/op |
21657 ns/op |
14378 ns/op |
1.51 |
Benchmark_Rewrite/Next_always_true (github.com/gofiber/fiber/v3/middleware/rewrite) |
556.1 ns/op 24 B/op 2 allocs/op |
360.7 ns/op 24 B/op 2 allocs/op |
1.54 |
Benchmark_Rewrite/Next_always_true (github.com/gofiber/fiber/v3/middleware/rewrite) - ns/op |
556.1 ns/op |
360.7 ns/op |
1.54 |
Benchmark_Bind_URI (github.com/gofiber/fiber/v3) |
3593 ns/op 768 B/op 32 allocs/op |
2349 ns/op 768 B/op 32 allocs/op |
1.53 |
Benchmark_Bind_URI (github.com/gofiber/fiber/v3) - ns/op |
3593 ns/op |
2349 ns/op |
1.53 |
Benchmark_Ctx_Cookie (github.com/gofiber/fiber/v3) |
205.9 ns/op 0 B/op 0 allocs/op |
135.7 ns/op 0 B/op 0 allocs/op |
1.52 |
Benchmark_Ctx_Cookie (github.com/gofiber/fiber/v3) - ns/op |
205.9 ns/op |
135.7 ns/op |
1.52 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheckSimple (github.com/gofiber/fiber/v3) |
68.96 ns/op 8 B/op 1 allocs/op |
44.71 ns/op 8 B/op 1 allocs/op |
1.54 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheckSimple (github.com/gofiber/fiber/v3) - ns/op |
68.96 ns/op |
44.71 ns/op |
1.54 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheck (github.com/gofiber/fiber/v3) |
76.16 ns/op 8 B/op 1 allocs/op |
50.32 ns/op 8 B/op 1 allocs/op |
1.51 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheck (github.com/gofiber/fiber/v3) - ns/op |
76.16 ns/op |
50.32 ns/op |
1.51 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheckAllowPrivate (github.com/gofiber/fiber/v3) |
81.49 ns/op 8 B/op 1 allocs/op |
52.33 ns/op 8 B/op 1 allocs/op |
1.56 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheckAllowPrivate (github.com/gofiber/fiber/v3) - ns/op |
81.49 ns/op |
52.33 ns/op |
1.56 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheckAllowAll (github.com/gofiber/fiber/v3) |
103.8 ns/op 8 B/op 1 allocs/op |
66.56 ns/op 8 B/op 1 allocs/op |
1.56 |
Benchmark_Ctx_IsProxyTrusted/WithProxyCheckAllowAll (github.com/gofiber/fiber/v3) - ns/op |
103.8 ns/op |
66.56 ns/op |
1.56 |
Benchmark_Router_Handler_Unescape (github.com/gofiber/fiber/v3) |
181.6 ns/op 0 B/op 0 allocs/op |
116 ns/op 0 B/op 0 allocs/op |
1.57 |
Benchmark_Router_Handler_Unescape (github.com/gofiber/fiber/v3) - ns/op |
181.6 ns/op |
116 ns/op |
1.57 |
Benchmark_Compress_Levels/Brotli_LevelBestCompression (github.com/gofiber/fiber/v3/middleware/compress) - B/op |
17 B/op |
11 B/op |
1.55 |
Benchmark_Communication_Flow (github.com/gofiber/fiber/v3) |
81.98 ns/op 0 B/op 0 allocs/op |
49.1 ns/op 0 B/op 0 allocs/op |
1.67 |
Benchmark_Communication_Flow (github.com/gofiber/fiber/v3) - ns/op |
81.98 ns/op |
49.1 ns/op |
1.67 |
This comment was automatically generated by workflow using github-action-benchmark.
93c1fcb to
706ae86
Compare
706ae86 to
5fad032
Compare
Reduce CI wall-clock per commit without changing results, output or coverage. test.yml: - add a concurrency group that cancels superseded in-progress PR runs while never cancelling pushes to main - compute coverage only on the ubuntu/1.25.x job (the only one that uploads it); the other 5 matrix combinations skip the coverage instrumentation - shard the repeated job into 3 parallel matrix shards of -count=5 each (15 shuffled runs total, with -race). Measured: ~9m41s -> ~3m49s. The job is dominated by fixed test latency (sleeps/timeouts), not CPU, so trimming -race alone did not help; splitting the repetitions does. benchmark.yml: - add the same PR-only concurrency cancellation The benchmark job stays a single job on purpose. It cannot be sharded on GitHub-hosted runners: the regression comparison relies on a cpu-model cache key so it only compares same-CPU results. Spreading benchmarks across multiple runners with heterogeneous CPUs breaks that guarantee and produces false alerts (a no-code-change PR showed a uniform ~1.5-2.5x ns/op "regression" with identical B/op and allocs/op, i.e. pure CPU variance). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5fad032 to
a55fb1f
Compare
Motivation
Reduce CI wall-clock per commit without changing results, output or coverage.
Key finding: the Test workflow's long pole is the
repeatedjob, and it is dominated by fixed test latency (133time.Sleepcalls: 14x3s, 17x1s, ...), not CPU. Removing-racealone saved only ~33s. The effective lever is parallelism.Changes
test.ymlmain).ubuntu/1.25.xjob (the only one that uploads it); the other 5 matrix combinations skip-coverprofile -covermode=atomic.repeatedjob sharded into 3 parallel matrix shards of-count=5each = still 15 shuffled runs total (3 distinct shuffle seeds),-racekept.benchmark.ymlMeasured impact (real run, not estimate)
repeatedFunctionality preserved
ubuntu/1.25.x.repeatedkeeps-race.Why the benchmark job is NOT sharded
An attempt to shard benchmarks across 4 runners cut wall-clock to ~4min but failed the comparison: a no-code-change commit reported a uniform ~1.5-2.5x
ns/op"regression" with identicalB/op/allocs/op, i.e. pure CPU variance. The benchmark comparison relies on acpu-modelcache key so it only compares same-CPU results; spreading work across GitHub's heterogeneous runners breaks that guarantee. Benchmark therefore stays a single job (correct over fast).🤖 Generated with Claude Code