Skip to content

test: resolve flaky integration test timing issues#3503

Merged
tippmar-nr merged 3 commits intomainfrom
test/flaky-integration-test-timing
Mar 24, 2026
Merged

test: resolve flaky integration test timing issues#3503
tippmar-nr merged 3 commits intomainfrom
test/flaky-integration-test-timing

Conversation

@tippmar-nr
Copy link
Copy Markdown
Member

@tippmar-nr tippmar-nr commented Mar 24, 2026

Summary

  • GCSamplerV2 tests: Add GC.Collect(0) calls so V2 per-generation subtraction reliably reports non-zero Gen0 collections on fast-starting CI processes.
  • OTel metrics tests: Wait for OTLP export log (every 5s) instead of analytic_event_data (every 60s), eliminating a timeout race.
  • OTel stress tests: Replace 60s analytics wait + 45s Thread.Sleep with a wait for 3 OTLP exports, cutting exercise time from ~123s to ~20s.

Test plan

  • Run OpenTelemetryMetricsTests suite — all variants should pass without 60s timeout flakiness
  • Run OpenTelemetryStressTests suite — should complete in ~30s instead of ~2.5min
  • Run AgentMetrics suite — GCSamplerV2 tests should pass consistently

🤖 Generated with Claude Code

- GCSamplerV2 tests: add Gen0-only GC.Collect(0) calls so the V2
  per-generation subtraction logic reliably reports non-zero Gen0
  collections regardless of startup timing.

- OTel metrics tests: wait for the OTLP export log line (fires every
  5s) instead of analytic_event_data (fires at ~60s), which races
  with the 60-second WaitForLogLine timeout on slow CI machines.

- OTel stress tests: replace the 60s analytics wait + 45s Thread.Sleep
  with a wait for 3 OTLP exports (~15s), cutting exercise time from
  ~123s to ~20s.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tippmar-nr tippmar-nr marked this pull request as ready for review March 24, 2026 15:22
@tippmar-nr tippmar-nr requested a review from a team as a code owner March 24, 2026 15:22
@tippmar-nr tippmar-nr enabled auto-merge (squash) March 24, 2026 15:48
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.82%. Comparing base (77679dd) to head (b3f8a03).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3503      +/-   ##
==========================================
+ Coverage   81.78%   81.82%   +0.03%     
==========================================
  Files         508      508              
  Lines       34220    34220              
  Branches     4040     4040              
==========================================
+ Hits        27986    27999      +13     
+ Misses       5268     5254      -14     
- Partials      966      967       +1     
Flag Coverage Δ
Agent 82.81% <ø> (+0.04%) ⬆️
Profiler 71.75% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tippmar-nr tippmar-nr merged commit 5cee30e into main Mar 24, 2026
117 checks passed
@tippmar-nr tippmar-nr deleted the test/flaky-integration-test-timing branch March 24, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants