Seed all random number generators in testing - #773
Merged
Conversation
tommyod
marked this pull request as draft
October 7, 2025 12:01
tommyod
marked this pull request as ready for review
October 8, 2025 10:25
lucaeg
self-requested a review
October 8, 2025 12:29
lucaeg
approved these changes
Oct 8, 2025
lucaeg
left a comment
lucaeg
left a comment
Contributor
There was a problem hiding this comment.
This looks good to me :) I only have two questions:
- Why use only one seed in some tests and a collection of seeds in others?
- How does using seeds exclude bad RNGs (that lead to test failures), do you mean that if a test fails for a particular seed then we will exclude that seed?
Contributor
Author
|
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Using RNG without seeds causes tests to fail every now and then. To test statistical properties, it is better to see the rng and test on many seeds. That way we can replicate across machines and we don't have to be stopped by false positives (what looks like test failures are just a bad rng).