Skip to content

Precision loss during round-trip of ppc_fp128 hexadecimal floating-point constants (double-double arithmetic) #124

@mewmew

Description

@mewmew

There is a known precision loss when parsing and printing LLVM IR assembly containing ppc_fp128 hexadecimal floating-point constants. This issue was identified by @dannypsnl, and the cause has been researched by @dannypsnl (see #31 (comment)) and @scottshotgg (see #31 (comment)).

Below follows a proof of concept illustrating the precision loss for the ppc_fp128 constant 0xM400C0000000000300000000010000000. From #31 (comment)

The problem happened, in this case, was because the sum of high and low is not exact, I create an example for it:

package main

import (
	"fmt"
	"math/big"
	"math"
)

func main() {
	precision := uint(106)
	// Operate on numbers of different precision.
	var z big.Float
	x := big.NewFloat(math.Float64frombits(0x0000000010000000)).SetPrec(precision)
	y := big.NewFloat(math.Float64frombits(0x400C000000000030)).SetPrec(precision)
	z.SetPrec(precision)
	z.Add(x, y)
	fmt.Printf("x = %.10g (%s, prec = %d, acc = %s)\n", x, x.Text('p', 0), x.Prec(), x.Acc())
	fmt.Printf("y = %.10g (%s, prec = %d, acc = %s)\n", y, y.Text('p', 0), y.Prec(), y.Acc())
	fmt.Printf("z = %.10g (%s, prec = %d, acc = %s)\n", &z, z.Text('p', 0), z.Prec(), z.Acc())
}
// Result:
// x = 1.326247369e-315 (0x.8p-1045, prec = 106, acc = Exact)
// y = 3.5 (0x.e000000000018p+2, prec = 106, acc = Exact)
// z = 3.5 (0x.e000000000018p+2, prec = 106, acc = Below)

Link to failing test case:

"0xM400C0000000000300000000010000000",

go test github.com/llir/llvm/ir/constant -run TestNewFloatFromStringForPPCFP128

Changing the precision of the math/big.Float from 106 (which is the precision of double-double arithmetic) to 1048 fixes this issue and resolves the loss of precision (see #31 (comment)). However, simply changing the precision to 1048 does not seem like the right fix, since it will affect the results of future arithmetic on these constants and may result in more memory usage and loss of performance.

Leaving this issue open in the hopes that someone has the expertise needed to figure out what is going wrong, and come up with a solution.

ref to sister issue #31.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions