Skip to content

Fix float parsing precision in SimpleQueryParser#4913

Merged
kgrgreer merged 6 commits intodevelopmentfrom
float-fix
Mar 14, 2026
Merged

Fix float parsing precision in SimpleQueryParser#4913
kgrgreer merged 6 commits intodevelopmentfrom
float-fix

Conversation

@abdelaziz-mahdy
Copy link
Copy Markdown
Collaborator

Problem

SimpleQueryParser corrupted small float values (e.g., 0.001, 0.0001) during parsing. The digits grammar action applied parseInt() to the fractional part of floats, stripping leading zeros — 0.001 became 0.1, 0.0001 became 0.1. This broke all float-based queries (IN RANGE, equality, comparison) for values with leading zeros after the decimal point.

Separately, Constant.toMQL() used toFixed(20) which added floating-point noise to the output (e.g., 0.1 became 0.10000000000000000555).

Solution

Add a rawDigits grammar symbol (identical grammar to digits but without the parseInt action) and use it in the float rule. This preserves the raw digit string "001" during float parsing instead of converting it to 1. Fix applied to both JS and Java parsers.

Fix Constant.toMQL() to use String() (shortest exact decimal representation) with toFixed(20) fallback only for scientific notation.

Changes

  • Add rawDigits grammar symbol in both JS and Java SimpleQueryParser to preserve leading zeros in float fractional parts
  • Fix Constant.toMQL() float formatting: String() instead of toFixed(20) to avoid floating-point noise
  • Add small-float precision tests to SimpleQueryParserTest, SimpleQueryParserJavaTest, and SimpleQueryParserMQLTest

@@ -191,8 +191,11 @@ foam.CLASS({

digits: str(repeat(range('0', '9'), null, 1)),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abdelaziz-mahdy why are we duplicating the grammar here. Shouldn't we have digits just call sym(rawDigits)?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noted: fixed

@kgrgreer
Copy link
Copy Markdown
Owner

The Java version has the same issue:
g.addSymbol("float",
new Seq1(1,
g.sym("ws"),
new Join(new Seq(
new Optional(Literal.create("-")),
g.sym("digits"),
new Optional(new Join(new Seq(Literal.create("."), new Optional(g.sym("digits")))))))));

digits now delegates to sym('rawDigits') instead of duplicating the
grammar rule. rawDigits holds the base grammar, digits adds parseInt.
@abdelaziz-mahdy
Copy link
Copy Markdown
Collaborator Author

The Java version has the same issue: g.addSymbol("float", new Seq1(1, g.sym("ws"), new Join(new Seq( new Optional(Literal.create("-")), g.sym("digits"), new Optional(new Join(new Seq(Literal.create("."), new Optional(g.sym("digits")))))))));

fixed.

@kgrgreer kgrgreer merged commit 0aa9f57 into development Mar 14, 2026
1 check passed
@Nauna Nauna deleted the float-fix branch March 16, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants