balancer/randomsubsetting: Extend the unit tests in the randomsubsetting package. #8781
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8781 +/- ##
==========================================
- Coverage 83.47% 82.43% -1.04%
==========================================
Files 419 414 -5
Lines 32595 32751 +156
==========================================
- Hits 27208 26999 -209
- Misses 4017 4095 +78
- Partials 1370 1657 +287 🚀 New features to boost your workflow:
|
marek-szews
changed the title
| _ "google.golang.org/grpc/balancer/roundrobin" // For round_robin LB policy in tests | ||
| ) | ||
|
|
||
| func (s) TestSubsettingEndpointsDomain(t *testing.T) { |
There was a problem hiding this comment.
This test is similar to the existing test TestCalculateSubset_Simple.
You can remove this and add more test cases in TestCalculateSubset_Simple if you think of any.
There was a problem hiding this comment.
+1
Yes, the only possible case that could be added to TestCalculateSubset_Simple is the case where the number of endpoints is strictly greater than the subset size.
There was a problem hiding this comment.
Indeed it is only one test case missing, i had renamed function and added to test set.
| } | ||
| } | ||
|
|
||
| func (s) TestUniformDistributionOfEndpoints(t *testing.T) { |
There was a problem hiding this comment.
Add a descriptive comment for this test.
There was a problem hiding this comment.
Move this test to the existing test file randomsubsetting_test.go.
There was a problem hiding this comment.
+1 to both the above comments. Please add comments about why the math is the way it is in the test.
There was a problem hiding this comment.
Done. Appropriate commentary added.
| } | ||
| } | ||
|
|
||
| func (s) TestUniformDistributionOfEndpoints(t *testing.T) { |
There was a problem hiding this comment.
Can you run the test ~10k times on your local to verify that the test is not flaky.
There was a problem hiding this comment.
How often does this flake currently? We do not want to add tests that can flake. We want a test that is guaranteed to pass every time that it runs, and when it fails, it should really mean that there is a bug in the code.
|
|
||
| endpoints := makeEndpoints(16) | ||
| expected := iteration / len(endpoints) * subsetSize | ||
| diff = expected / 7 // allow ~14% difference |
There was a problem hiding this comment.
Can you explain the reasoning behind 14% error-rate.
There was a problem hiding this comment.
Can you look at this and modify the test to calculate how many iterations are needed for statistical significance, instead of arbitrarily adding huge error rate to satisfy small range of iterations.
There was a problem hiding this comment.
I carefully analyzed the test acceptance criteria and came to the conclusion that it was too naive. Finally I use the Chi-square goodness-of-fit test, standard statistical method used to validate whether a dataset follows a uniform distribution. It assesses if the observed frequencies in your data align with the expected frequencies of a theoretical distribution where every outcome has an equal chance of occurring.
3 participants
Add more unit tests to increase test coverage of 'randomsubsetting' package.
RELEASE NOTES: