conformance: decode schema 1 and schema 3 in one binary

[bgm-malbeclabs]

Summary of Changes

Makes one dz-conformance binary decode schema 1 and schema 3, so the per-venue version pin can be deleted.

• wire.SupportedSchemas / SchemaSupported replace ExpectedSchemaVersion: a feed now has a set of accepted versions (midpoint {1}, everything else {1, 3}). Schema 2 stays rejected — no publisher ever deployed it, and a layout nobody has seen on the wire is a guess.
• New engine/instrdef.go holds the (feed, schema) → InstrumentDefinition offset table. That message is the only part of the wire format whose layout moves between the supported versions, so one table covers multi-schema support for the whole engine. expectedMsgLen and the field accessors read it; no offset literal survives outside it.
• engine.beginFrame's downgrade gate changes from schemaVersion > <the one expected version> to membership. Ordering was a proxy for "can we decode this" that only worked for a one-element set — and it was already wrong for schema 2, which is unsupported but lower than 3, so it graded such a stream at full severity through schema-3 offsets. beginFrame's own comment named multi-version decode as the proper fix; this is it.
• VERSIONING.md records the exception this creates. The spec says a decoder MUST reject a Schema Version it was not built for; this validator now does not, deliberately and for itself only. An undocumented exception to a MUST is how the rule quietly stops meaning anything.

Why. dz_conformance_version is pinned per venue today because a build reads one schema. Hyperliquid emits 1, Kalshi emits 3, and the wrong pin mis-sizes InstrumentDefinition by 50 bytes and fires MSG.LENGTH_PER_TYPE — a must rule — on every definition datagram. The pin is not configuration; it is the same knowledge held both on the wire and in a group_vars string a human has to keep equal to it. malbeclabs/infra's own dz_conformance.yml comment already documents where that breaks: a fourth Kalshi metro added the way dub was inherits a pin with no feeds.

Schema-1 offsets came from git tag top-of-book/v1.0.0, not from any current spec file — the 80-byte layout was retired by the 2.0.0 and 3.0.0 releases. The code comment says so, because the source of truth for half that table is a tag rather than HEAD.

The branch also carries the design spec (docs/superpowers/specs/2026-09-09-fleet-conformance-design.md) this implements §2 of, and its plan. Happy to split those into a separate PR if you would rather review them apart from the code.

Spec Impact

• No wire-format change. No field layout, message type, Magic or size is altered. No Schema Version byte is bumped.
• PATCH (editorial) for VERSIONING.md — the new subsection describes an existing tool and clarifies the scope of a rule without changing required behaviour, which is the class table's own definition of editorial.
• No sibling spec or the Reference Data supplement is affected.
• The tools/conformance change is tool behaviour, not specification, and takes a MINOR release on its own conformance/<version> tag line: it adds accepted versions and removes none.

Review Notes

Not in this PR, and deliberately. The per-venue pins in malbeclabs/infra are what this unblocks, but deleting them means cutting a release and rolling production, which belongs in that repo with a human driving it. No release should be cut from any commit before 37be4a0 — between the first commit and that one, the decoder accepts schema 1 while still reading schema-3 offsets, which grades hyperliquid worse than today.

Verify against a real capture before releasing. Replay one hyperliquid (schema 1) and one Kalshi (schema 3) capture through the new binary and diff the JSON reports against the two pinned binaries they replace. A new violation there is a finding to investigate, not a regression to suppress — it may be a real publisher fault the wrong pin was masking.

Known gaps, all judged non-blocking by review:

• core/ruledoc.go's accepted-set text and wire.SupportedSchemas can drift; guarded by a comment naming the other, not a compile-time check. Package engine already imports both core and wire, so a test asserting the rule text names every entry needs no new import edge.
• The reject rule is still stated unqualified in top-of-book/spec.md, midpoint/spec.md, market-by-price/spec.md, README.md, and market-by-order/spec.md:798 ("A subscriber MUST reject…") — which bites, because dz-conformance's README calls it "a conformance subscriber". Left alone here: each spec is independently versioned, so amending them is five PATCH releases and a call for their owners.
• wire/wirebuild/build.go takes supported[len-1] as "highest", an unstated ordering contract on SupportedSchemas.

One behaviour change beyond the schema work, flagged because it is easy to miss in the diff: refdata.go now skips an InstrumentDefinition whose body is too short instead of feeding zeros into onInstrumentDef. That removes cascaded false REFDATA.STALE_SEQ_TAG_AFTER_BUMP findings; MSG.LENGTH_PER_TYPE still fires on the short message.

MBP Price Bound is deliberately still unread. The layout table has a valid offset for it, but the pre-existing switch sent MBP to its default: arm, so reading it now would be a live grading change smuggled into a schema change. Review confirmed it is inert either way today — MBP's payloads are 0x40–0x42 and reach none of the three call sites that consume priceBound.

Testing Verification

• Schema-3 grading is unchanged, traced path by path: expectedMsgLen at schema 3 returns the same lengths, instrDefAllFields returns the same four values the old switch did for every feed, beginFrame classifies schema-3 non-midpoint and midpoint-at-3 exactly as > did, and wirebuild's default resolves to the same version. Schema 3 is what every production venue except hyperliquid emits.
• TestSchema1TOBStreamRaisesNoMustViolation runs two hand-built schema-1 frames through the real wire.Decode → engine.Process/Flush/EndRun path and asserts no must violation. Proven non-vacuous by mutation: forcing the schema-1 lookup to return the schema-3 layout fails it with MSG.LENGTH_PER_TYPE type 0x02: length 80, expected 130 — the exact production symptom the two pinned builds exist to avoid.
• The offset table was mutation-tested: +1 on ManifestSeq and PriceBound together (so the 50-byte delta test still passes) fails 18+ tests.
• Schema-1 offsets independently re-derived from tag top-of-book/v1.0.0 by two reviewers, agreeing with the table and with the test fixture — four derivations of a layout that exists in no current spec file.
• TestInstrDefLayoutManifestSeqIsLast pins the ordering invariant the bounds-check discards rely on, enumerating rows through instrDefLayoutFor itself rather than a parallel list, with a floor assertion so an empty sweep fails loudly instead of passing vacuously.
• TestExpectedMsgLenIsSchemaAware sweeps the whole uint8 type space per non-midpoint feed rather than a hand-listed set, so a type that becomes schema-dependent later cannot slip through.
• TestNonconformantMBPCapture golden counts unchanged at every step. The bundled capture is 2001/2001 schema-1 frames and now decodes through correct offsets rather than mismatched ones.
• go build, go vet, gofmt -l and go test -count=1 ./... clean across all seven packages, run with the build cache cleared.

🤖 Generated with Claude Code