Skip to content

feat(sleep): add correctness-first annotation case study - #3

Open
viranovskaya wants to merge 1 commit into
mainfrom
feat/sleep-case-study
Open

feat(sleep): add correctness-first annotation case study#3
viranovskaya wants to merge 1 commit into
mainfrom
feat/sleep-case-study

Conversation

@viranovskaya

Copy link
Copy Markdown
Collaborator

Why

Before a shared benchmark contract is designed, the repository needs a small correctness-first case showing that HED search can retrieve parallel sleep annotations by meaning while preserving their source. The case should also make clear where semantic retrieval ends and ordinary sleep analysis begins.

What changed

  • Add a 14-row synthetic fixture modeled on the CC0 BOAS three-stream annotation structure: human consensus, PSG-based AI, and headband-based AI.
  • Define independent numeric-label oracles for N2 and unavailable-data states, then verify that Basic, String, and Object HED search return the expected rows and onsets.
  • Enforce the fixture contract, including 30-second epochs, allowed source-specific codes, unique monotonic onsets, complete sidecar mappings, and warning-sensitive HED validation.
  • Generate deterministic correctness JSON and a readable report.
  • Document the boundary between HED retrieval and downstream alignment, disagreement, transition, and agreement analyses.

No annotation stream is treated as ground truth. This PR follows the current per-case repository structure and exposes a callable run_case() that a future shared CLI can wrap; it does not define the universal benchmark JSON contract.

Verification

  • Two independent correctness and report generations were byte-identical.
  • All three HED search engines matched the independent expected rows and onsets for every query.
  • Adversarial fixtures with an unknown stage code, a non-30-second duration, or a missing sidecar mapping were rejected.

Relates to #1.

@viranovskaya
viranovskaya marked this pull request as ready for review September 2, 2026 15:31
@viranovskaya
viranovskaya requested a review from VisLab September 2, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant