Float precision changes when loading JSON function-call data through datasets / pandas

[firefighter-eric]

Checklist

• I have searched existing issues, and this is a new bug report.

Bug Description

Hi ms-swift team,

I found a float precision issue when loading function-call training data.

The original JSONL contains values like:

{"value": 0.7}

but after ms-swift loads the data through datasets / pandas, it may become:

{"value": 0.7000000000000001}

This is problematic for Function Call SFT because the model learns the exact token sequence, so these two representations are different training targets.

It seems that the JSON data is parsed into floating-point values by pandas and then serialized again, introducing floating-point artifacts.

It would be helpful if function-call arguments could preserve their original JSON representation, or avoid unnecessary float conversion during dataset loading.

Thanks!

How to Reproduce

Prepare a Function Call training sample where the assistant calls a tool with a float argument:

{
  "messages": [
    {
      "role": "user",
      "content": "Set the zoom level to 0.7."
    },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "set_zoom",
            "arguments": "{\"zoom\": 0.7}"
          }
        }
      ]
    }
  ]
}

After loading the dataset with ms-swift, the assistant tool-call argument may become:

{
  "arguments": "{\"zoom\": 0.7000000000000001}"
}

---

检查清单

• 我已经搜索过现有的 issues，确认这是一个新的 bug report。

Bug 描述

你好，ms-swift 团队，

我发现 Function Call 训练数据在加载过程中存在浮点精度变化的问题。

原始 JSONL 中的数据例如：

{"value": 0.7}

但 ms-swift 通过 datasets / pandas 加载后，可能会变成：

{"value": 0.7000000000000001}

这对于 Function Call SFT 会有影响，因为模型学习的是精确的 Token 序列，这两种表示实际上是不同的训练目标。

目前看起来是 JSON 数据经过 pandas 转换成浮点数后，再重新序列化时引入了浮点误差。

希望 Function Call arguments 能尽量保留原始 JSON 表示，或者避免数据加载阶段不必要的浮点类型转换。

谢谢！

如何复现

准备一条 Function Call 训练数据，其中 assistant 调用 Tool，并包含浮点参数：

{
  "messages": [
    {
      "role": "user",
      "content": "将变焦设置为 0.7。"
    },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "set_zoom",
            "arguments": "{\"zoom\": 0.7}"
          }
        }
      ]
    }
  ]
}

使用 ms-swift 加载数据后，assistant 的 Tool Call 参数可能变成：

{
  "arguments": "{\"zoom\": 0.7000000000000001}"
}

[firefighter-eric]

@claude help

[Linxiushen]

I can reproduce this, and the root cause is upstream in 🤗 datasets, not in ms-swift's own code.

swift/dataset/loader.py loads local JSON/JSONL via datasets.load_dataset("json", ...). Inside datasets/packaged_modules/json/json.py, whenever the file cannot go through the fast pyarrow reader — e.g. field= is used, or a batch has rows with mixed/absent struct fields such as messages[].tool_calls present in one row and absent in the next — it falls back to pandas_read_json(), which calls pd.read_json(...) without precise_float=True. pandas' default (fast, imprecise) float parser is what turns 0.7 into 0.7000000000000001:

>>> import io, pandas as pd
>>> pd.read_json(io.StringIO('{"v": 0.7}\n'), lines=True)["v"][0]
np.float64(0.7000000000000001)
>>> pd.read_json(io.StringIO('{"v": 0.7}\n'), lines=True, precise_float=True)["v"][0]
np.float64(0.7)

Same with 0.3 → 0.30000000000000004. End-to-end with datasets 4.8.4 / pandas 3.0.6, a JSONL whose first row has tool_calls[].function.arguments = {"zoom": 0.7} and whose second row is a plain assistant message loads as {'zoom': 0.7000000000000001}; when the tool-call arguments are later re-serialized into the training text, that artifact ends up in the target tokens.

Notes:

• If arguments is kept as a JSON string in the JSONL ("arguments": "{\"zoom\": 0.7}"), pandas never parses the number and the value survives intact — that's a workaround until the upstream fix lands.
• The fix in datasets is a one-liner (precise_float=True in pandas_read_json); I'm preparing a PR there and will link it here.

(Analysis and reproduction done with Claude Code, AI-assisted.)

[Linxiushen]

Upstream fix submitted: huggingface/datasets#8652 (precise_float=True in pandas_read_json, with a regression test for the field= path). Until it is released, keeping arguments as a JSON string in the JSONL avoids the re-parse.

[hjh0119]

Would quoting the value avoid this issue?

-"arguments": "{\"zoom\": 0.7}"
+"arguments": "{\"zoom\": "0.7"}"

[firefighter-eric]

Would quoting the value avoid this issue?

-"arguments": "{"zoom": 0.7}"
+"arguments": "{"zoom": "0.7"}"

Thanks. Quoting 0.7 changes the parameter from a number to a string, which does not match the tool schema. Even when arguments is already a JSON string, ms-swift parses it into an object, and the precision issue occurs when reading the preprocessing cache.
A verified workaround is SWIFT’s native tool_call format with the entire content stored as a JSON string. Model-specific tool formatting still works, and the value remains 0.7.