Skip to content

fix(docreader): Excel 百分比被读小 100 倍 —— 加开关(默认关) - #4423

Merged
lyingbug merged 2 commits into
Tencent:mainfrom
linus-liu-web:fix/docreader-xlsx-percent-as-text
Oct 17, 2026
Merged

lyingbug merged 2 commits into
Tencent:mainfrom
linus-liu-web:fix/docreader-xlsx-percent-as-text

Conversation

@linus-liu-web

Copy link
Copy Markdown
Contributor

问题

Excel 里的百分比是显示格式,不是存储值:单元格实际存 0.61123366,靠 0.00% 渲染成 61.12%。openpyxl / pandas 只读存储的分数、丢掉格式,于是用户看到的 61.12% 到流水线里变成 0.6112 —— 100 倍偏差。在财务报表上,模型会把错的数照抄进它写的东西里。

(真实案例:一个财报里 A45:A46 / D45:D46 跨两行合并,同一行里同时出现 0.55% 和 0.005509433890047721。)

修法

fill_merged_cells_xlsx 现在把带百分比格式的数值单元格实体化成表格显示的字面文本(0.6112 → "61.12%",格式置 General),在同一趟 openpyxl 遍历里完成 —— 且在填合并单元格之前。

顺序很重要:合并区域只有 master 单元格带 number_format,被覆盖的格子一律是 General;先填合并会把裸分数复制到第 2..n 行,那几行就整片漏掉。

为什么只处理百分比

货币和千分位改变的是数字怎么显示,不是它是什么(1,234,567 和 1234567 是同一个数),重写它们只是折腾单元格、没有语义收益。日期格式 openpyxl 已经转成 datetime。

开关与默认值

默认关 —— 因为它把一个单元格从数值变成字符串,任何对它排序或聚合的代码会看到不同的输入:

  • 部署级:DOCREADER_XLSX_PERCENT_AS_TEXT=false(compose .env),每次调用读取;
  • 单次上传 / 按知识库:parser_engine_overrides.xlsx_percent_as_text(true/false),优先于环境变量 —— 与既有的 pdf_force_scanned 覆盖同一条优先级。

开关关闭时,XLSX 预处理路径与现状逐字节相同,存量部署读每个工作簿的方式完全不变。

开启时的语义变化:百分比单元格变成字符串("61.12%"),不再是浮点数 —— 下游要对这些格子做计算就得自己解析。这正是它 opt-in 的原因。

另含一个独立开关(同样默认关)

DOCREADER_XLSX_PCT_ROW_HINT_MIN=N 可以救回"自己丢了百分比格式、但同一行还有别的百分比格"的单元格(真实案例:折旧率 0.0592 格式是 0.00_ ;[Red]\-0.00\ )。它会改写没有格式标注的值,所以默认 0(关),每命中一次打一行 INFO 便于审计。

测试

docreader/tests/test_xlsx_percent_format.py(15 例,夹具是用 openpyxl 生成的真实 .xlsx)。

改前 (pristine): Ran 15 tests ... FAILED (failures=15)
改后 (本 PR)   : Ran 15 tests ... OK

覆盖:默认关逐字节一致、61.12% 实体化、合并行顺序坑、货币/普通数字/日期不被误伤、开关优先级、opt-in 兜底。

未覆盖

老式二进制 .xls(走 xlrd 引擎,不经过这个预处理步骤)。

Excel keeps a percentage as the stored fraction plus a display format: the
cell holds 0.61123366 and "0.00%" renders it as 61.12%. openpyxl and pandas
read only the fraction, so a cell the user sees as 61.12% reaches the
pipeline as 0.6112 — a 100x error that a model then copies verbatim into
whatever report or bid is built from a financial statement.

Add an opt-in switch (DOCREADER_XLSX_PERCENT_AS_TEXT, or the per-upload
parser_engine_overrides.xlsx_percent_as_text; off by default) that rewrites
percentage-formatted numeric cells as the text the spreadsheet displays, in
the same openpyxl pass that fills merged cells and *before* that fill: only
the master cell of a merge carries a number_format while the covered cells
are General, so filling first would leave rows 2..n as bare fractions and the
same row would show both "0.55%" and 0.005509433890047721.

Only percentages are rewritten — currency and thousands separators do not
change what a number means ("1,234,567" and 1234567 are the same number) and
date formats are already converted to datetimes by openpyxl. A separate
opt-in (DOCREADER_XLSX_PCT_ROW_HINT_MIN, default 0=off) can recover a cell
that lost its percentage format while its row kept one.

With the switch off the XLSX pre-parse path is byte-identical to the previous
behavior, so existing deployments read workbooks exactly as before.

@lyingbug lyingbug left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复核此前评审结论及最新提交,相关 CI 检查全部通过,同意合入。

@lyingbug
lyingbug merged commit a9290dc into Tencent:main Oct 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants