Skip to content

conformance: score a capture's opening partial group unverifiable, not 642 must-violations #65

Description

@ben-dz

Sibling of #53 (same rule, opposite end of the capture). Measured against two preserved live Phoenix MBP captures, on bdz/adhoc-388 (i.e. with #64's journal fix already applied).

The artifact

A pcap of a running publisher's snapshot port always opens mid-group: the rotation is continuous, so the first frames carry SnapshotLevels whose SnapshotBegin was emitted before the recorder started. Every one of those levels scores MBP.SNAP.GROUP_STRUCTURE (must) "SnapshotLevel with no open SnapshotBegin", and the run exits 1 on a feed that did nothing wrong.

Measured, first snapshot-port frames of each capture (message types decoded; 0x42 = SnapshotLevel, 0x20 = SnapshotBegin):

  • mbp-conf-20260901T1453Z.pcap: frames seq 343496–343501 carry 37+12+37+14+12+25 = 137 bare 0x42; the first 0x20 appears at seq 343502. Reported violations on this capture: exactly 137.
  • mbp-conf-20260902T1543Z.pcap: frames from seq 98509 likewise all-0x42 until the first 0x20. Reported violations: 642, all inside seqs 98509–98526.

So in both captures the entire violation count is the opening partial group, and nothing else on the rule fires.

Why it matters beyond the exit code

The tool already forgives the capture's tail — a group left open when the file ends scores unverifiable{truncated}. The head deserves the symmetric treatment, and for the same reason #63 gives for recorder-admitted loss: an artifact of the archive must not be charged to the publisher.

It also actively misleads. The 137 above were read as live evidence for a publisher defect (malbeclabs/phoenix#163, "137 violations in one SOL burst") — the count matches the head artifact exactly, and #53's hardcoded instrument id 0 is what made it read as SOL. The code defect #163 named was independently real and is fixed (malbeclabs/phoenix#166), but its live evidence was this artifact, which also means that fix's abort path has never been exercised by a real capture.

Suggested shape

Grade orphan SnapshotLevels Unverifiable with a distinct reason (e.g. capture_start) only before the first SnapshotBegin observed for that (channel, port); after that, an orphan is a real violation and must stay one — that is exactly #53's post-reset tail and #163's shape. A test needs both halves: head orphans forgiven, and a mid-capture orphan burst after a reset still scored a violation. Trimming the pcap head is not a fix: a trim re-creates the cut.

Acceptance for both preserved captures once this and #53 land: zero GROUP_STRUCTURE violations, exit 0, with the forgiven levels visible in unverifiable_by_reason rather than absent.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions