Skip to content

Epic: deserialize from the event stream without composing a tree #49

Description

@elioseverojunior

Tracking epic for deserializing typed targets straight from the parser event stream, without composing an intermediate Node tree.

Why this is the priority

Measured on a dedicated host, glaucus beats every other Rust YAML library on 37 of 39 head-to-head comparisons. Both losses are deserialize, both against the same implementation: 1.18× on the medium fixture and 1.20× on large. It does not lose on small, on any serialize benchmark, or to anyone else.

The measurement also points at the cause:

opt-level = "z" opt-level = 3
deserialize, small 1.10× (glaucus ahead) 0.95×
deserialize, medium 1.03× 0.85×
deserialize, large 1.00× — exact tie 0.83×

A constant-factor lead scales equally for both sides. Being amplified by the optimiser is what happens when one side has less work to optimise away. The competing implementation walks parser events straight into the target; glaucus::from_str calls compose_one_versioned, builds a Node tree, then deserialises from it.

Closing this turns 37/39 into 39/39.

Approach

A second Deserializer over the event stream, beside the Node-based one rather than replacing it:

  • from_str::<T> streams for typed targets.
  • from_str::<Value> keeps its existing TypeId fast path into the tree — Value is a tree, and streaming into one would compose it anyway.
  • from_str_node is unchanged.

Both engines share exactly one thing: the resolvers in glaucus_core::schema (#47). Nothing else.

The risk, and what is done about it

Two implementations of anchor resolution that must agree is the same shape that produced three divergent scalar resolvers — see #40, where glaucus schema validate and glaucus from_str reached opposite verdicts on 0x1F.

So the differential test harness is task 1, not task 9. It compares both paths across a 20-case corpus — anchors, nested anchors, merge keys, tags, %YAML 1.1, malformed input — and every subsequent task gates on it. It runs green from the start, while there is still only one engine, so the first disagreement it ever reports is a real one rather than a backlog discovered at integration.

Tasks

Sequencing

#50  (harness — green before there is anything to diverge)
 └─ #51  (tape)
     └─ #52  (scalars)
         └─ #53  (collections)
             └─ #54  (anchors — gates on the harness)
                 └─ #55  (merge keys)
                     └─ #56  (limits)
                         └─ #57  (wire from_str)
                             └─ #58  (measure)

Strictly sequential. Each task's tests need the previous one's machinery.

Performance constraints

Requirements, not aspirations. A task that passes its functional test and breaks one of these is not done.

  • No allocation on the anchor-free path. Most documents have no anchors; the tape starts recording lazily at the first one.
  • Scalars reach serde borrowed. EventKind::Scalar carries a Cow<'a, str> into the input. Where borrowed, visit_borrowed_str — copying forfeits the reason to stream.
  • Replay by index range over one flat tape, not a Vec per anchor, so a nested anchor's events are stored once.

Acceptance

Streaming must be measurably faster on the medium and large fixtures with non-overlapping confidence intervals — significance, not point estimates.

If it is not, this is reverted. A second engine costs maintenance forever, and the differential suite that keeps two engines honest is itself a permanent cost. The decision belongs at #58, on evidence, rather than after the code has accumulated dependents.

Depends on

#38 — specifically #47 (shared resolvers, which both engines must use) and #45 (Value::get_path, used by the anchor tests).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestepicTracking issue spanning multiple issuesparityCross-implementation parity worktype:feature

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions