Fix too restrictive condition in Java RyuFloat impl. #244
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When `e2 < 0`, current `RyuFloat` calculates `dvIsTrailingZeros` by the code below, which should be equivalent to `mv%2^(q-1) == 0`. ```java dvIsTrailingZeros = (q < FLOAT_MANTISSA_BITS) && (mv & ((1 << (q - 1)) - 1)) == 0; ``` https://github.com/ulfjack/ryu/blob/1264a946ba66eab320e927bfd2362e0c8580c42f/src/main/java/info/adams/ryu/RyuFloat.java#L207 If I understand correctly, `(q < FLOAT_MANTISSA_BITS)` is an optimization that works as follows: - The maximum value of `v` is `2^FLOAT_MANTISSA_BITS - 1` - Therefore, when `q` is greater than or equal to `FLOAT_MANTISSA_BITS`, `v` is never divisible by `2^q` - So, if `(q < FLOAT_MANTISSA_BITS)` is false, we can skip the `(mv & ((1 << (q - 1)) - 1)) == 0` check **However, I think it should be `(q - 1 < FLOAT_MANTISSA_BITS + 2)` because**: - We're checking if `mv` is divisible by `2^(q-1)`, not `2^q`. So the left-hand side should be `q - 1` - The maximum value of `mv` is larger than `2^FLOAT_MANTISSA_BITS - 1`: - `mv` is `4 * m`, where `m` is the 23-bit mantissa. This means `mv <= 4 * (2^23 - 1) = 2^25 - 4` - To check if `mv` is divisible by `2^(q-1)`, the condition `q < 23` is too restrictive, because `mv` can be divisible by up to `2^24` - Therefore, the right-hand side should be `FLOAT_MANTISSA_BITS + 2` --- Actually, a test from `f2s_test.cc`'s `lotsOfTrailingZeros` fails in the Java implementation: https://github.com/ulfjack/ryu/blob/1264a946ba66eab320e927bfd2362e0c8580c42f/ryu/tests/f2s_test.cc#L69 ``` 1) lotsOfTrailingZeros(info.adams.ryu.RyuFloatTest) org.junit.ComparisonFailure: expected:<0.002441406[2]> but was:<0.002441406[3]> ``` This is because: - `mv = 41943040`, `e2 = -34`, and `q = 23` - Therefore, `dvIsTrailingZeros = false` because `q < 23` is false (but it should be true) - By step 4, `dv` is reduced from `244140625` to `24414062` (with `lastRemovedDigit = 5`) - The condition `(dvIsTrailingZeros && (lastRemovedDigit == 5) && (dv % 2 == 0))` evaluates to false (but it should be true!), because `dvIsTrailingZeros = false` https://github.com/ulfjack/ryu/blob/1264a946ba66eab320e927bfd2362e0c8580c42f/src/main/java/info/adams/ryu/RyuFloat.java#L267-L269 As a result, it is rounded up to `24414063`, but it should be rounded to `24414062`.
tanishiking
force-pushed
the
fix-ryu-float-java
branch
from
dbf9759 to
ab78548
Compare
tanishiking
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix #243