-
Notifications
You must be signed in to change notification settings - Fork 28
Fix wrong infinite precision for number types with prefixed units #1594
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix wrong infinite precision for number types with prefixed units #1594
Conversation
3550500 to
316641f
Compare
wsafonov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general looks good, only couple of minor points:
- naming: not sure if
constantis the best way to describe the range with min == max bounds. When I seeisConstant(), I think about a range that cannot be modified. Something likeisSingular()orisSinglePoint()would be better imho. - in
isContant(), checking both bounds are not equal NegInf / PosInf is covered by the last condition, since NegInf != PosInf. For safety, I would rather check min.isNotEmpty && max.isNotEmpty - even if this normally wouldn't happen, that definitely should not count as a single point range.
Not related to this bug, but I just observed that 2 kbyte + 300 bit gets accepted as number, since both of them are digital information units. But from the precision perspective that doesn't make sense. Not sure what would be the right approach here - compute everything as bits or even not allowing mix of byte/bits.
wsafonov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Important points are all addressed, looks good now.
Yes, at this location the chunk only imports |
Will squash and then merge. |
Details: - add failing tests showing the problem with infinite precisions - compute proper precision for number types with prefixed unit - update CHANGELOG
9415804 to
c2fd0ac
Compare



This PR extends some tests for physunits to reproduce the problem of infinite precision for types of prefixed units. It provides the fix, and updates the CHANGELOG accordingly.
This solves #1593.