Commit 5c45d43
committed
test(qr-auth): replace flaky max-deviation bias check with chi-square
The short-code distribution test asserted that no base62 character
deviated more than 15% from its expected count. That statistic is the
maximum over 62 correlated near-normal cells, so its tail is fat: at
n=36000 the per-cell relative SD is ~4.1%, which puts the 15% bound at
|z| ~ 3.65, and taken as a max over 62 cells it fires on a perfectly
uniform generator about 1.6% of the time. Measured directly: 48 spurious
failures in 3000 simulated runs. It had been rerun-to-green repeatedly
and most recently red-herringed a PR review.
Chi-square is the correct test for "is this multinomial uniform", and
unlike 0.15 its threshold is derivable. df=61, Wilson-Hilferty puts the
p=1e-6 critical value at ~129, so the bound is 130.
Power is unchanged. Removing rejection sampling from generateShortCode
reintroduces modulo bias (256 % 62 = 8, so eight characters draw five
chances per 256 instead of four) and was verified against the real code
in an isolated worktree: chi-square 243.06 against the 130 limit. The
threshold sits in a wide empty gap, 3000 clean runs peaked at 104 while
200 biased runs bottomed out at 174.5.
Also iterate the alphabet explicitly rather than the observed keys, so a
character that never appears counts as zero instead of being skipped.
Verified: 30/30 consecutive runs of the real test pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 57b6be1 commit 5c45d43
1 file changed
Lines changed: 31 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
250 | 250 | | |
251 | 251 | | |
252 | 252 | | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
253 | 256 | | |
254 | 257 | | |
255 | 258 | | |
| |||
265 | 268 | | |
266 | 269 | | |
267 | 270 | | |
268 | | - | |
269 | | - | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
274 | 297 | | |
275 | 298 | | |
276 | | - | |
277 | | - | |
278 | | - | |
| 299 | + | |
| 300 | + | |
279 | 301 | | |
280 | 302 | | |
281 | 303 | | |
| |||
0 commit comments