Skip to content

Port tools/knoodle_io.hpp parsing to Tools::InString (measured 5.5x on coordinate input) #36

Description

@designbynumbers

tools/knoodle_io.hpp predates the fast I/O now in the library. Its parsing path is std::getline per line, then std::istringstream per line, then std::stod per token -- allocating, locale-aware and throwing. Tools::InString reads the whole buffer once and parses with std::from_chars.

Measured

200,000 lines of 3-column floats (6.1 MB), -O3 -march=native, same checksum from both:

path time
getline + CleanLine + istringstream + stod (what ReadKnot does today) 56.8 ms
Tools::InString + from_chars 10.3 ms

5.5x. Floating-point std::from_chars is available on this toolchain, so FromChars<T> takes the fast path rather than its stod fallback.

This matters most for 3D coordinate input, which is the large-input case: a few hundred thousand vertices is an ordinary polygon file, and the parser is currently a real fraction of the run.

What is already delegated

ReadKnot hands the PDC-native format (u <color> / s <flag> rows) straight to PDC_T::FromInString, so that path is already yours. It is the 3/4/5/6/7-column TSV path that is still hand-rolled.

What the port has to preserve

Not a drop-in replacement -- CleanLine does input munging that InString has no reason to provide, and it is load-bearing for files people actually have:

  • % starts a comment, anywhere in a line, with no escaping;
  • commas and braces become separators, so Mathematica-style {1, 2, 3} reads;
  • CRLF input must work (a surviving \r breaks the last token on a line);
  • column-count detection (3 = geometry, 4/5/6/7 = PD codes) keys on the value count and on whether any token contained a ..

Those are pinned by test/knoodle_io_check (73 checks, 2 ms), added in d30352ea16ace55f22406a272f4816197f601740, so the port has a regression net before it starts. That test also records three leniencies of the current implementation that a rewrite would want to decide about deliberately rather than inherit or drop by accident:

  • std::stod stops at the first non-numeric character, so 5abc is accepted as 5;
  • has_float is a literal search for ., so 1e5 does not set it -- a five-column line containing 1e5 is classified as a signed PD code rather than refused;
  • CleanLine turns each brace and comma into its own space, so {1, 2, 3} comes back double-spaced.

from_chars would change the first two by construction: it reports where it stopped, so trailing garbage becomes detectable, and it handles exponents without a separate has_float heuristic. Both are arguably improvements, but they are behaviour changes and should be chosen rather than fallen into.

No urgency -- filing so it is not forgotten.

Prepared with Claude Code (Opus 5)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions