Fail PRs based on Rust test statistics #5678
Description
Security issue notifications
If you discover a potential security issue in s2n we ask that you notify
AWS Security via our vulnerability reporting page. Please do not create a public github issue.
Problem:
Our Rust tests output the time taken to compile and run. It should be easy for us to parse that output and fail a PR if it increases the time taken. See this stack overflow question.
Note that the option --timings is used to report compile times, not test timings. That really confused me for a while.
Need By Date:
N/A
Solution:
New CI job that runs the tests and parses test stats to fail if time increases. Note that we need to do this in s2n-quic as well, I just didn't want to make duplicate issues for both repos.
Requirements / Acceptance Criteria:
I think it's better for us to fail on this per PR rather than passively just tracking test times.
Out of scope:
Is there anything the solution will intentionally NOT address?