dvyukov declextract speedup #5602
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
dvyukov
commented
Dec 12, 2024
dvyukov
commented
- pkg/clangtool: cache combined output
- pkg/ifaceprobe: optimize cache
- tools/syz-declextract: parallelize
Instead of caching output for each file separately, cache total combined output in a single file. Caching output for each file is not useful in practice, I either use everything cached, or regenerate whole cache. Caching combined output is much more efficient. With function info there are lots of duplication across individual output files. E.g. I am getting 6GB cache for individual files, and only 60MB for the combined cache. Also change how caching works. Remove the flag and always use the cache if it exists. It's much more convinient and safer to use (accidentially not using the cache). The cache file can be removed to force regeneration.
Instead of storing real PC values store indexes into the PCs table. This significantly reduces size of the cache (in my case from 1823 MB to 473 MB) and actually makes use of the cache simpler (don't need separate map).
Do kernel probing, source code analysis and loading of syscall rename map in parallel. Also change probe caching to the scheme we now use for the clang tool cache so the same reasons.
a-nogikh
approved these changes
Dec 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.