Skip to content

Commit 7828bb8

Browse files
committed
perf(ab_test): ignore noisy network and MMDS metrics on flaky instances
Add m5n.metal (al2 host), m8g.metal-{24,48}xl, and m8i.metal-{48,96}xl to the IGNORED list for network latency and MMDS A/B tests respectively. These instance/metric combinations exhibit high variance (up to 67% swings in both directions) on unrelated commits, causing persistent false-positive pipeline failures. The noise on m8g correlates with the May 2 AL2023 rootfs update and affects only guest kernel linux-6.1 dimensions. MMDS noise on m8i is bidirectional within the same build. The tests still run and collect data on these instances; they just no longer block the pipeline. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
1 parent 81236d8 commit 7828bb8

1 file changed

Lines changed: 12 additions & 0 deletions

File tree

tools/ab_test.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,18 @@ def format_with_reduced_unit(value, unit):
135135
}
136136
for instance in ["m8i.metal-48xl", "m8i.metal-96xl"]
137137
],
138+
# Network latencies on m5n.metal w/ al2 host
139+
{"instance": "m5n.metal", "performance_test": "test_network_latency", "host_kernel": "linux-5.10"},
140+
# Network latencies on m8g
141+
*[
142+
{"instance": instance, "performance_test": "test_network_latency"}
143+
for instance in ["m8g.metal-24xl", "m8g.metal-48xl"]
144+
],
145+
# MMDS metrics on m8i
146+
*[
147+
{"instance": instance, "performance_test": "test_mmds_performance"}
148+
for instance in ["m8i.metal-48xl", "m8i.metal-96xl"]
149+
],
138150
# block latencies if guest uses async request submission
139151
{"fio_engine": "libaio", "metric": "clat_read"},
140152
{"fio_engine": "libaio", "metric": "clat_write"},

0 commit comments

Comments
 (0)