We sadly have various flaky tests that fail from time to time (often with multiple months in between them). Ideally we would fix them properly, but that might be a lot of work for comparatively small benefit. Simply disabling the tests is also suboptimal as most of these tests are still generally useful, even if they sometimes fail.

The goal of this ticket is to consistently annotate the counterexample output of such tickets, briefly explaining (or linking to an issue) that a failure of this test might just be flaky, and not due to any recent changes.

Examples:

• Mempool benchmark is flaky #313
• False positive on not-pipelined block in ThreadNet #545
• Handle flakiness in BlockchainTime.Simple #567
• Triage new Byron ThreadNet failure #582
• mock-test Praos ThreadNet failure: unexpected message delay #1105
• Flaky Byron ThreadNet failure: ImmutableDB is leaking file handles #1294
• Adversarial chains win if checked with relaxed parameters is inherently flaky #1442

This is might also be a good opportunity to make sure that these test failures actually still happen, and have not been fixed inadvertently in the mean time.