Description
The following table lists the tests that have failed on the master branch over the last couple months. These tests can be assumed to be flaky.
This data was generated by a periodic script that parses Action run logs and extracts the tests that failed. Links to the logs in GitHub Actions is included but log retention is short (I think 1 month) so will not be available for all instances.
The data below includes action runs since 2025-02-17. Last updated 2025-04-25.
Fixed tests:
- TestNativeHistogramFuzz (thanks @SungJin1212)
- TestSchedulerShutdown_FrontendLoop (thanks @SungJin1212)
- TestRecoverAlertsPostOutage
| Name | Number of Failures | Links |
|---|---|---|
| TestRing_ShuffleShardWithLookback_CorrectnessWithFuzzy | 11 | 0 1 2 3 4 5 6 7 8 9 10 |
| TestQuerierWithBlocksStorageRunningInSingleBinaryMode | 9 | 0 1 2 3 4 5 6 7 8 |
| TestMinimizeSpreadTokenGenerator | 6 | 0 1 2 3 4 5 |
| TestPrometheusCompatibilityQueryFuzz | 6 | 0 1 2 3 4 5 |
| TestNativeHistogramFuzz | 6 | 0 1 2 3 4 5 |
| TestRecoverAlertsPostOutage | 2 | 0 1 |
| TestQuerierWithBlocksStorageRunningInMicroservicesMode | 2 | 0 1 |
| TestExperimentalPromQLFuncsWithPrometheus | 2 | 0 1 |
| TestBackwardCompatibilityQueryFuzz | 2 | 0 1 |
| TestExpandedPostingsCacheFuzz | 2 | 0 1 |
| TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod | 2 | 0 1 |
| TestCompactor_RingLifecyclerShouldAutoForgetUnhealthyInstances | 2 | 0 1 |
| TestCompactor_ShouldCompactOnlyShardsOwnedByTheInstanceOnShardingEnabledWithShuffleShardingAndMultipleInstancesRunning | 1 | 0 |
| TestSchedulerShutdown_FrontendLoop | 1 | 0 |
| TestSortLabels | 1 | 0 |
| TestLoader_ShouldCacheIndexNotFoundOnBackgroundUpdates | 1 | 0 |
| TestDisableChunkTrimmingFuzz | 1 | 0 |
| TestCompactor_ShouldCompactOnlyUsersOwnedByTheInstanceOnShardingEnabledAndMultipleInstancesRunning | 1 | 0 |