sprt: alternative initial p value in normalized LLR calculation

[87flowers]

Fixes #879.

@vdbergh: Are there any good alternatives here? As one possible alternative to this, what are your thoughts on using the MLE of the distribution given the expectation as an initial value for determining the MLE of the distribution for the t-value?

[vondele]

Thanks, practically that seems to work for me!
Examples of various skill levels playing master:

--------------------------------------------------
Results of sf0 vs sf1 (3+0.03, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -997.10 +/- nan, nElo: -4339.56 +/- 54.52
LOS: 0.00 %, DrawRatio: 0.00 %, PairsRatio: 0.00
Games: 156, Wins: 0, Losses: 155, Draws: 1, Points: 0.5 (0.32 %)
Ptnml(0-2): [77, 1, 0, 0, 0], WL/DD Ratio: -nan
LLR: -2.95 (-100.3%) (-2.94, 2.94) [-1000.00, -500.00]
--------------------------------------------------

or

--------------------------------------------------
Results of sf0 vs sf1 (3+0.03, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -600.97 +/- 118.46, nElo: -1409.98 +/- 53.17
LOS: 0.00 %, DrawRatio: 0.00 %, PairsRatio: 0.00
Games: 164, Wins: 0, Losses: 154, Draws: 10, Points: 5.0 (3.05 %)
Ptnml(0-2): [72, 10, 0, 0, 0], WL/DD Ratio: -nan
LLR: -2.97 (-100.7%) (-2.94, 2.94) [-1000.00, -500.00]
--------------------------------------------------

or

--------------------------------------------------
Results of sf0 vs sf1 (3+0.03, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -472.36 +/- 70.82, nElo: -946.75 +/- 48.89
LOS: 0.00 %, DrawRatio: 1.03 %, PairsRatio: 0.00
Games: 194, Wins: 1, Losses: 171, Draws: 22, Points: 12.0 (6.19 %)
Ptnml(0-2): [74, 22, 1, 0, 0], WL/DD Ratio: inf
LLR: -2.99 (-101.7%) (-2.94, 2.94) [-1000.00, -500.00]
--------------------------------------------------

or

--------------------------------------------------
Results of sf0 vs sf1 (10+0.1 - 3+0.03, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -350.40 +/- 66.12, nElo: -594.00 +/- 59.27
LOS: 0.00 %, DrawRatio: 7.58 %, PairsRatio: 0.00
Games: 132, Wins: 5, Losses: 106, Draws: 21, Points: 15.5 (11.74 %)
Ptnml(0-2): [40, 21, 5, 0, 0], WL/DD Ratio: inf
LLR: 8.31 (282.2%) (-2.94, 2.94) [-1000.00, -500.00]
--------------------------------------------------

[robertnurnberg]

Nice. Unrelated changes to the PR, but can you make these two edits from #881 as well:

Line 208 change comment to

// https://www.cantate.be/Fishtest/comparing_approximations.pdf

And move the computation of sigma in line 266 outside the loop.

PS: I am wondering if a similar fix should be applied to the LLR calculation on Fishtest.

[vondele]

Still seemingly getting stuck on this:

--------------------------------------------------
Results of sf0 vs sf1 (10+0.1 - 3+0.03, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -269.37 +/- 26.21, nElo: -456.25 +/- 30.45
LOS: 0.00 %, DrawRatio: 12.80 %, PairsRatio: 0.00
Games: 500, Wins: 27, Losses: 352, Draws: 121, Points: 87.5 (17.50 %)
Ptnml(0-2): [109, 108, 32, 1, 0], WL/DD Ratio: 4.33
LLR: 0.00 (0.0%) (-2.94, 2.94) [-1000.00, -500.00]
--------------------------------------------------

[vdbergh]

I am sorry for not reacting. I am at a place where I can't seriously think. I will try to reply later tonight.

[87flowers]

Still seemingly getting stuck on this:

--------------------------------------------------
Results of sf0 vs sf1 (10+0.1 - 3+0.03, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -269.37 +/- 26.21, nElo: -456.25 +/- 30.45
LOS: 0.00 %, DrawRatio: 12.80 %, PairsRatio: 0.00
Games: 500, Wins: 27, Losses: 352, Draws: 121, Points: 87.5 (17.50 %)
Ptnml(0-2): [109, 108, 32, 1, 0], WL/DD Ratio: 4.33
LLR: 0.00 (0.0%) (-2.94, 2.94) [-1000.00, -500.00]
--------------------------------------------------

I think I'm coming around to the idea of bailing out with an approximation if we aren't successful; but I'm happy to wait and see what Michel comes up with.

[vdbergh]

I think it is indeed best to bail out when the standard algorithm fails, and to use an approximation. The approximation LLR_normalized_alt() in the source code would be good.