Add benchmark workflow #153
Open
+682
−459
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
reyhankoyun
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
reyhankoyun
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #153 +/- ##
=======================================
Coverage 91.88% 91.88%
=======================================
Files 14 14
Lines 2404 2404
Branches 2404 2404
=======================================
Hits 2209 2209
Misses 147 147
Partials 48 48 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
simonmarty
reviewed
Dec 11, 2025
reyhankoyun
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
reyhankoyun
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
reyhankoyun
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
reyhankoyun
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
simonmarty
reviewed
Dec 17, 2025
simonmarty
previously approved these changes
Dec 17, 2025
Signed-off-by: Simon Marty <[email protected]>
simonmarty
approved these changes
Dec 17, 2025
simonmarty
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
simonmarty
requested changes
Dec 17, 2025
Contributor
simonmarty
left a comment
simonmarty
left a comment
There was a problem hiding this comment.
It looks like the upload is not succeeding with
Warning: No files were found with the provided path: aws_secretsmanager_caching/target/criterion/. No artifacts will be uploaded.
…results aren't generated
reyhankoyun
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
ThirdEyeSqueegee
approved these changes
Jan 6, 2026
ThirdEyeSqueegee
requested changes
Jan 6, 2026
Member
ThirdEyeSqueegee
left a comment
ThirdEyeSqueegee
left a comment
There was a problem hiding this comment.
The Criterion book has a section about benchmarking async functions. Did we take this into consideration when writing these tests? Not sure how it plays with Tokio.
Comment on lines
+55
to
+57
| // Warm up the cache. | ||
| rt.block_on(cache.get_secret_value("secretid", None, None, false)) | ||
| .unwrap(); |
Member
There was a problem hiding this comment.
Not sure we need to do this. Criterion seems to warm up caches automatically (ref)
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: benchmark-results-${{ github.sha }} |
Member
There was a problem hiding this comment.
Criterion has built-in support for identifying perf improvements and regressions, it does this by checking older results in the benchmark results folder. We should use the same folder for all benchmark results (or perhaps per-branch folders) (ref).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
This PR builds off of PR #122 and creates a benchmark workflow similar to integration tests workflow that runs the benchmarks added in PR #122
Add two basic benchmarks
Using criterion (used by the AWS SDK for Rust).
This hopefully will serve to catch performance regressions in future code changes and open the door to profiling.
Successful run from fork: https://github.com/reyhankoyun/aws-secretsmanager-agent/actions/runs/20082405961/job/57612354790
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.