Fix benchmark CI #609
Merged
Fix benchmark CI #609
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yngve793
approved these changes
Mar 27, 2025
achaikou
force-pushed
the
fix_benchmark_ci
branch
from
bfaf408 to
16430c3
Compare
achaikou
commented
Mar 27, 2025
There was a problem hiding this comment.
Not sure how to test that the two wheels are different.
It got tested for us when I was doing datasource changes 😅
I was very surprised there was no reduction in performance when I was running new benchmarks against slower datasources 😄
Currently you can only check that different wheels are installed.
Run benchmarks manually in Actions and supply older ref, for example c946c3c.
You'll have
Successfully installed segyio-1.9.12
vs
Successfully installed segyio-1.9.13
I've also added prints to assure where segyio is loaded from.
Don't think more could be done.
Current logic is flawed. Intention of benchmark tests was to build library from two different commits. However only C/C++ library was built separately, python code was not touched and included. When current commit's directory was checked out to run the fresh tests, so was other python code: tests were always run on the same python part of the library. To fix that, build a local wheel from existing code and make sure it is being used instead of local code.
achaikou
force-pushed
the
fix_benchmark_ci
branch
from
16430c3 to
3ca438f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current logic is flawed.
Intention of benchmark tests was to build library from two different commits. However only C/C++ part was built separately, python code was not touched.
When current commit's directory was checked out to run the fresh tests, so was other python code: tests were always run on the same python part of the library.
To fix that, build a local wheel from existing code and make sure it is being used instead of local code.