fix: Turtle serializer abbreviates typed literals without validating lexical forms (#772)

[jeswr]

🚧 DRAFT — Note: this is an agent-generated draft PR (part of the agent-assisted triage announced in #823). It is for @jeswr to review — please do not merge or mark ready-for-review until he has signed off.

Fixes #772. Part of the triage programme tracked in #824.

Problem

atomicTermToN3 (src/serializer.js) abbreviates xsd:integer / xsd:decimal / xsd:double / xsd:boolean literals to native Turtle tokens based only on the datatype URI, without checking that the lexical form is valid for that datatype. Originally reported for booleans in #772, generalized to the whole class after review:

Input literal	Before	After
"true"^^xsd:boolean	false — value flipped (the #772 report)	true
"false" / "1" / "0" ^^xsd:boolean	false / true / false	unchanged
"yes"^^xsd:boolean (invalid)	false — value lost	"yes"^^xsd:boolean
"abc"^^xsd:integer (invalid)	bare abc — invalid Turtle, datatype lost	"abc"^^xsd:integer
"1.2.3"^^xsd:decimal (invalid)	1.2.3 — invalid Turtle	"1.2.3"^^xsd:decimal
"2."^^xsd:decimal (valid XSD; Turtle needs a digit after the dot)	2. — invalid Turtle	2.0
"NaN" / "INF" / "-INF" ^^xsd:double (valid XSD; no Turtle DOUBLE form)	NaN.0e0 etc. — mangled	"NaN"^^xsd:double etc.
"abc"^^xsd:double (invalid)	abc.0e0 — mangled	"abc"^^xsd:double

Fix

Each of the four abbreviation cases now abbreviates only when the lexical form is valid for the datatype and expressible as a Turtle token:

• xsd:integer — /^[+-]?[0-9]+$/
• xsd:decimal — /^[+-]?(?:[0-9]+\.?[0-9]*|\.[0-9]+)$/, then normalized so the token has a digit after the dot (5 → 5.0, 2. → 2.0)
• xsd:double — /^[+-]?(?:[0-9]+\.?[0-9]*|\.[0-9]+)(?:[eE][+-]?[0-9]+)?$/ (excludes INF/-INF/NaN), then the existing ./e normalization
• xsd:boolean — true/1 → true, false/0 → false

Anything else falls through to the existing verbose "value"^^datatype path, so serialization is lossless and always emits valid Turtle. Diff: src/serializer.js +17/−1; everything else is tests.

Ecosystem backward-compatibility assessment

• Serializer-only — parsing is untouched (read lenient, write strict).
• Valid, correctly-serialized forms are byte-identical. No tests/serialize/*-ref.* golden fixture changes; the full fixture corpus serializes identically (CI test:serialize green on this head).
• Output changes only where the previous output was wrong — the table rows above: a flipped value, lost data, or non-conformant Turtle. Every changed output is valid Turtle and round-trips with value and datatype preserved.
• Who the Turtle serializer mis‑normalizes xsd:boolean literal "true" as "false" #772 flip actually hit: booleans stored with the standard lexical form true — e.g. minted by applications via lit('true', undefined, XSD.boolean), or imported from RDF/JS-ecosystem libraries (N3.js et al.) that keep the canonical lexical form. Booleans from rdflib's own Turtle parser or Literal.fromBoolean are stored as '1'/'0' and always serialized correctly, so SolidOS/NSS round-trips through rdflib's own parser are unaffected.
• Related: Boolean literals parsed as "0", "1" #147 (the parser normalizes true/false to '1'/'0' on parse) is untouched here — but both lexical conventions now serialize to correct tokens.
• Semver: no API change; suggest release-patch (pure bug fix), though release-minor is defensible if any serializer-output delta is treated as behavioural.

Tests

Extends the existing tests/unit/serialize-test.js (no new file):

• booleans: valid forms (true/false/1/0) unchanged; invalid forms (yes, TRUE, 2, "", 01) serialize verbosely
• new integer/decimal/double blocks: valid forms abbreviate exactly as before; invalid forms serialize verbosely
• serialize→parse round-trip tests asserting value + datatype are preserved for every invalid lexical form (no data loss)

CI is green on this head (Node 22.x / 24.x: unit + serialize golden fixtures + type tests + typedoc).

Relationship to #830

#830 (term-level serialization rework) supersedes this PR and subsumes this fix at its term layer. This PR stands as the minimal, self-contained alternative if the larger rework is not taken — whichever lands, the other should be closed or rebased accordingly.

---

@jeswr — over to you for review; leaving as draft.

[jeswr]

Note: agent-generated update, for @jeswr to review.

Addressed the review comment by broadening the fix from booleans to the whole class of datatype-abbreviation bugs: in atomicTermToN3, native Turtle abbreviation of xsd:integer / xsd:decimal / xsd:double / xsd:boolean literals is now guarded by lexical-form validation. Invalid (or Turtle-inexpressible) lexical forms fall through to the verbose "value"^^datatype serialization — no data loss, no invalid Turtle.

Concretely fixed:

• "yes"^^xsd:boolean no longer flips to false; "TRUE", "2", "", "01" also go verbose
• "abc"/"1.5"/"0x10"^^xsd:integer no longer emit bare invalid tokens
• "1.2.3"/"."/"1e5"^^xsd:decimal go verbose; valid-XSD-but-not-Turtle "2." normalizes to 2.0
• "NaN"/"INF"/"-INF"^^xsd:double (valid xsd:double, no Turtle DOUBLE form) and "abc" no longer emit NaN.0e0-style garbage

Valid lexical forms serialize exactly as before. Tests: per-datatype native/verbose assertions plus serialize→parse round-trips asserting value+datatype preservation for every invalid form (30 new tests; unit suite 338 passing, full npm test + npm run lint clean).

[jeswr]

LGTM.

This should be flagged on the linkeddata/rdflib, solidOs and discourse channels before merging in case this affects consumers of the library in unexpected ways.

May need to reserve for an mver.