Skip to content

Commit 479ee76

Browse files
authored
Change compiler regression report to median (#7648)
This change the logic to find the baseline of the compiler regression. before it find the max metric value from past 3-6 days as baseline value, and compare it to the latest 3 days metric values. however, this create some noisy alert. Change it to median value , to avoid some scenarios such as shown below <img width="393" height="201" alt="image" src="https://github.com/user-attachments/assets/edeff7f8-09a8-494e-8127-b3b0984a297f" />
1 parent eab4b80 commit 479ee76

File tree

1 file changed

+4
-4
lines changed
  • aws/lambda/benchmark_regression_summary_report/common

1 file changed

+4
-4
lines changed

aws/lambda/benchmark_regression_summary_report/common/config.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -251,25 +251,25 @@
251251
name="passrate",
252252
condition="greater_equal",
253253
threshold=0.9,
254-
baseline_aggregation="max",
254+
baseline_aggregation="median",
255255
),
256256
"geomean_speedup": RegressionPolicy(
257257
name="geomean_speedup",
258258
condition="greater_equal",
259259
threshold=0.95,
260-
baseline_aggregation="max",
260+
baseline_aggregation="median",
261261
),
262262
"compression_ratio": RegressionPolicy(
263263
name="compression_ratio",
264264
condition="greater_equal",
265265
threshold=0.95,
266-
baseline_aggregation="max",
266+
baseline_aggregation="median",
267267
),
268268
"compilation_latency": RegressionPolicy(
269269
name="compilation_latency",
270270
condition="less_equal",
271271
threshold=1.15,
272-
baseline_aggregation="min",
272+
baseline_aggregation="median",
273273
),
274274
},
275275
notification_config={

0 commit comments

Comments
 (0)