profile_diffs.py: script to compare Go coverage profiles, outputs new… #32
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… blocks This python script accepts two paths to Go coverage profiles and outputs the new code blocks that were hit in the second profile and not the first. If the script is used on both permutations of coverage profiles, it's possible to see whether coverage increased or decreased fairly easily. The script should be adapted in the future to: - use branch coverage (see something like: https://github.com/junhwi/gobco/) - automatically parse this output data and tell how many new branches were hit and how many old branches were not hit. - compare the number of hit counters instead of "hit" vs. "not hit". Help from ChatGPT 4o
Contributor
Author
|
Example output on #26 when comparing it to |
Contributor
Author
|
I think this needs some work to actually be useful for an end-user. |
Collaborator
|
Two questions before I go ahead and do a more thorough review.
|
Contributor
Author
|
For 1, yes it should be run both ways and ideally the output would be more helpful. Maybe it could summarize per file the branch difference. The tooling in llvm is way better than go tooling, I'm not sure if there's any sort of port or if we can fuzz with libfuzzer and lcov to get pretty coverage maps. |
|
@gijswijs: review reminder |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
… blocks
This python script accepts two paths to Go coverage profiles and outputs the new code blocks that were hit in the second profile and not the first. If the script is used on both permutations of coverage profiles (e.g.
<profile1> <profile2>in one run and<profile2> <profile1>in the other run), it's possible to see whether coverage increased or decreased fairly easily. The script should be adapted in the future to:Help from ChatGPT 4o
Usage:
python python_diffs.py <path_to_coverage_profile1> <path_to_coverage_profile2> <path_to_output_file>