Unmarshal to float32 yields different output than encoding/json

[bimoadityar]

Describe the bug

Unmarshaling some values to float32 yields different output than encoding/json.

To Reproduce

Run the following code:

import (
	"encoding/json"
	"testing"

	"github.com/bytedance/sonic"
	"github.com/stretchr/testify/assert"
)

func TestPrecision(t *testing.T) {
	b := []byte("7.328900098800659")
	var js float32
	var ss float32
	_ = json.Unmarshal(b, &js)
	_ = sonic.Unmarshal(b, &ss)
	assert.Equal(t, js, ss) // js == 7.3289, ss == 7.3289003
}

Expected behavior

For sonic to return output exactly the same as encoding/json.
I am guessing it is because sonic seems to unmarshal to float64 before casting the value to float32, which somehow seems to give less precision than parsing the value directly to float32.

Sonic version:

v1.14.2-0.20251010110621-72bd4caf6fb6

Environment:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bimoadityar/.cache/go-build"
GOENV="/home/bimoadityar/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/bimoadityar/.go/pkg/mod"
GOOS="linux"
GOPATH="/home/bimoadityar/.go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/bimoadityar/.config/spkit/bin/sdk/go1.23.12"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/bimoadityar/.config/spkit/bin/sdk/go1.23.12/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.23.12"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/bimoadityar/Documents/bimo-test-2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1204808054=/tmp/go-build -gno-record-gcc-switches"

[bimoadityar]

From further checking and asking Gemini, seems that the precision loss happens essentially because sonic does more rounding compared to encoding/json.

Here is a simplified code that shows this effect:

func TestPrecision2(t *testing.T) {
	b := "3.4028233649732406e+38"
	fj, _ := strconv.ParseFloat(b, 32) // basically what encoding/json is doing
	fs, _ := strconv.ParseFloat(b, 64) // basically what sonic is doing

	// Error: expected: 3.4028235e+38, actual: 3.4028233e+38
	assert.Equal(t, float32(fj), float32(fs))
}

Here is an explanation emitted by Gemini 3 Pro:

Explanation

The input string b := "3.4028233649732406e+38" is carefully chosen to be extremely close to the exact midpoint between the largest representable float32 (math.MaxFloat32) and the next smaller float32.

1. Exact Values:

   • MaxFloat32: 3.402823466...e+38 (Bit pattern: ...111)
   • Next Smaller Float32: 3.402823266...e+38 (Bit pattern: ...110)
   • Exact Midpoint: 3.40282336497324058...e+38 (Values exactly halfway round to the nearest "even" bit pattern).

   Your input string ends in ...406, which is slightly larger than the midpoint (...4058...).

2. ParseFloat(b, 32) (Direct to 32-bit):

   • The parser sees that the input is slightly larger than the midpoint.
   • It correctly rounds up to math.MaxFloat32.
   • Result: 3.4028235e+38

3. ParseFloat(b, 64) (Intermediate 64-bit):

   • The Midpoint itself is exactly representable as a float64.
   • The input is so close to the Midpoint that float64 precision cannot distinguish the tiny difference (the difference is smaller than the machine epsilon for float64 at this scale).
   • So, ParseFloat(b, 64) returns the Midpoint exactly.
   • When you cast this float64 Midpoint to float32, it is exactly halfway between the two float32 values.
   • Go uses Round-to-Even for ties. The "Next Smaller" float has an even bit pattern (...0), while MaxFloat32 has an odd one (...1).
   • It rounds down to the Next Smaller float.
   • Result: 3.4028233e+38

Summary

The 32-bit parse is "more correct" here because it sees the tiny fraction that pushes the value just past the midpoint. The 64-bit parse loses that tiny fraction, landing exactly on the midpoint, and the subsequent cast relies on the tie-breaking rule (round-to-even), which happens to go in the opposite direction.

---

So, from what I understand:

1. rounding loss happens twice in sonic: (1) when we call ParseFloat with bitsize 64, (2) when we cast the result into float32
2. rounding loss happens only once in encoding/json: (1) when we call ParseFloat with bitsize 32
   causing the encoding/json output to be closer than sonic in such kind of cases.

[bimoadityar]

Illustration from the sample input:

input                     : -7.328900098800659
parse64 output            : -7.32890009880065917968750  sign:1  exp:10000000001 mant:1101010100001100101100110000000000000000000000000000
parse32 output (json)     : -7.32889986038208007812500  sign:1  exp:10000001    mant:11010101000011001011001
parse64 cast to 32 (sonic): -7.32890033721923828125000  sign:1  exp:10000001    mant:11010101000011001011010

input                     : 3.4028233649732406e+38
parse64 output            : 340282336497324057985868971510891282432.0     sign:0  exp:10001111110 mant:1111111111111111111111010000000000000000000000000000
parse32 output (json)     : 340282346638528859811704183484516925440.0     sign:0  exp:11111110    mant:11111111111111111111111
parse64 cast to 32 (sonic): 340282326356119256160033759537265639424.0     sign:0  exp:11111110    mant:11111111111111111111110

Is it possible for sonic to parse directly to 32bit instead of going to float64 first?

[bimoadityar]

Hi @AsterDY @liuq19 , could kindly help check this when you have time? Thanks~