Fix and reorganize shrinking #1385
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The current shrinking implementation had a bug/limitation: it always tries to shrink the sender. In other words, if a transaction has a sender that is not minimal, Echidna will try to reduce it. However, the transaction might require a different sender to still cause the assertion failure, and the fact that Echidna will unconditionally try to lower it might cause the shrinking to get 'stuck'. This commit reworks the shrinking process to try and favor a more incremental shrinking, with multiple passes of smaller changes. The old process consisted of: - 50% chance of removing one tx - 50% chance of (reducing sender _and_ shrinking) _all_ txs The new process consists of, for each transaction independently: - 35% chance of shrinking the transaction - 30% chance of removing the transaction - 30% chance of keeping the transaction as-is - 5% chance of reducing sender "wait" calls are treated differently: - 60% chance of reducing or removing delay - 40% chance of keeping it as-is Fixes: #1224
genRandomR is unspecified if lo>hi
Contributor
Member
Author
|
Hi @BowTiedRadone! We merged #1399 to fix #1224 so that's no longer an issue. On the other hand, this is a larger refactor that kind of changes the approach to shrinking and that I feel might lead to better results. But as it is a significant change, it requires more experimentation and testing to find a sweet spot for the different weights and such, and to make sure it doesn't regress shrinking performance. You're more than welcome to play with the weights on some real life shrinking problems and report your observations if you want 😄 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current shrinking implementation had a bug/limitation: it always tries to shrink the sender. In other words, if a transaction has a sender that is not minimal, Echidna will try to reduce it. However, the transaction might require a different sender to still cause the assertion failure, and the fact that Echidna will unconditionally try to lower it might cause the shrinking to get 'stuck'.
This commit reworks the shrinking process to try and favor a more incremental shrinking, with multiple passes of smaller changes. The old process consisted of:
The new process consists of, for each transaction independently:
"wait" calls are treated differently:
Fixes: #1224