Skip to content

Apparent loss of percision when converting Rational -> double #28

@MReed2

Description

@MReed2

((double)Rational.Pow((Rational)0.95, 734)) != Math.Pow(0.95, 734)

Specifically, ((double)Rational.Pow((Rational)0.95, 734) == 4.4578585297200003E-17, while Math.Pow(0.95, 734) ==4.4578585297232747E-17. Wolfram Alpha agrees with Math.Pow, so I'm assuming that it is correct.

I suspect that the (double) conversion logic is converting to a decimal, then converting that to a double, because if you cast both to the decimal (instead of double) they do produce the same result.

Upon further examination, yeah, it looks like it is converting to decimal, then to double -- if you try to convert a small value into a double (which can handle smaller numbers than a decimal can), the underflow exception complain that "Value was either too large or too small for a Decimal." rather than the expected double.

Very odd problem -- I'd have assumed that decimal always has more significant digits than double, but apparently not (!?)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions