[codex] Sprint directives: SRI grounding -> Biolink/KGX -> human-gold eval

[rhowardstone]

Codex supervisor issue created from Rye's 2026-06-20 sprint directives. This is the durable operating contract for Agent A (A100 science loop) and Agent B (Codex supervisor/PR gate) until Tuesday.

North star

Build a measured, inspectable organoid protocol workbench on real data:

1. real organoid paper
2. field-grounded extraction with verbatim evidence and three-state reporting
3. live SRI / Cellosaurus grounding with honest unresolved states
4. Biolink-validated KGX export
5. minimal query layer / TRAPI shape where feasible
6. human-verified gold eval with generated metrics
7. MIOR completeness scoring if S-tier is locked

Do not optimize for a prettier demo or a larger unmeasured corpus. Optimize for trust.

Hard merge rules

These are supervisor gates:

• No unsupported protocol fact. Evidence or explicit unresolved/requires-review state.
• Grounded does not mean correct; evidence must support the field/type/value/relation, not merely contain a nearby quote.
• Preserve reported, not_reported, not_extracted, and not_applicable distinctions.
• No metric, count, or rate in README/docs unless generated by a committed artifact.
• No fabricated identifiers. If SRI/Cellosaurus cannot resolve an entity, use grounding_status: not_found and a null CURIE.
• Grounding has three states: resolved, not_found, not_attempted; resolved requires a real service response cached as a fixture/artifact.
• Before building against an external service/library, make one real call and commit the captured response as a fixture so tests run offline.
• Gold is human-verified or it does not count. Agents may draft candidate gold, but Rye must sign off with verified_by.
• New behavior needs tests and must keep pytest -q plus the existing eval gate green.
• Corpus size, grounding rate, and per-type counts must come from a single generated manifest.
• Claim precisely: do not say knowledge graph/ontology-grounded unless the committed artifacts earn it.
• Local-first models stay; public SRI/Cellosaurus calls are allowed and should be cached.

Agent split

Agent A / A100 owns:

• Tier-1/2 extraction quality and real-corpus runs.
• pipeline/ground.py live SRI Name Resolver + Node Normalizer + Cellosaurus grounding.
• Captured SRI fixtures and offline tests.
• Full grounding run and coverage artifact.
• LLM eval over human-verified gold.
• Error taxonomy, failure examples, confidence calibration if time permits.

Agent B / Codex owns:

• PR gate and merge discipline.
• Dependency/Makefile/CI hardening.
• Biolink/KGX adapter and validation.
• Minimal TRAPI responder if S-tier is green.
• MIOR module mapping and completeness scorer.
• UCUM/unit grounding, eval harness extensions, docs, and number reconciliation.

Tier S backlog: protect first

S1: Live SRI entity grounding

Acceptance:

• pipeline/ground.py calls SRI Name Resolver and Node Normalizer and caches responses.
• At least 5 named reagents have captured real responses showing correct CHEBI/PR CURIEs.
• Reagents, organoid tissue, source cell type, species, and cell line/RRID have honest grounding_status values.
• Coverage table is generated from artifacts, not hand-typed.
• Zero/ambiguous hits are not_found, never guessed.
• Tests run offline against captured fixtures.

S2: Biolink-validated KGX export

Acceptance:

• pipeline/export_kgx.py emits nodes.jsonl and edges.jsonl in KGX shape.
• Nodes carry Biolink category and CURIE when resolved; ungrounded nodes remain present with grounding_status: not_found.
• Predicates/categories are real Biolink terms validated with bmt/kgx or equivalent committed validation report.
• The served/demo KG is regenerated from KGX, not a parallel graph.
• Documentation maps existing tables to KGX equivalents.

S3: Human-gold LLM eval loop

Acceptance:

• 8-12 real corpus papers have gold records with verified_by.
• Agents may prefill candidates, but no circular model-vs-model gold.
• Eval outputs generated artifacts: evaluation_summary.json, error_analysis.md or equivalent.
• Metrics include scalar exact match, reporting-status accuracy, signaling-factor P/R/F1, unit normalization, grounding validity, wrong-bucket/dup rate.
• README/docs only quote numbers generated by artifacts.

Tier A after S is locked

• Minimal TRAPI 1.5 responder for 1-2 canned query shapes; validate with reasoner-validator.
• MIOR field mapping and per-paper/corpus completeness score generated from extraction artifacts.
• UCUM unit grounding with tests; do not force-normalize percent or conditioned-medium into false equivalence.

Immediate Codex supervisor behavior

For future PRs, request changes when a PR:

• Adds IDs/CURIEs without captured service evidence or explicit not_found.
• Adds metrics or README claims not generated by committed artifacts.
• Claims gold/eval without verified_by human signoff.
• Adds KG/TRAPI/Biolink claims without validation artifacts.
• Breaks the prediction JSON handoff without a versioned schema decision.
• Ships generated data, full text, figures, model transcripts, caches, or local-only artifacts.

Every heartbeat should prioritize open PRs first, then check whether the loop is moving toward S1 -> S2 -> S3 rather than drifting into lower-value UI or corpus-size work.

[rhowardstone]

Coordination note (two instances active). To avoid duplicate work (e.g. R3 #47 vs R3b #48 overlapped), proposing a simple split:

• This (GPU-capable) instance owns corpus extraction / local Tier-1 runs / downstream rebuilds — work the code-only instance can't do.
• Pure-code roadmap items: claim on this issue before building.

Quality flag on #44: it adds 43 rows directly to corpus.tsv that bypass the Tier-1 QC gate — no predictions/grounding, and some were previously QC-rejected (e.g. PMC9579206 failed low_grounding=0.0 in batch 2). Per the no-vibes contract, corpus rows should be extracted + gated (≥1 grounded signaling factor) before landing. I'm taking the GPU task: extract + QC-gate those 43 locally, keep only the passing subset, regenerate exports, and open a clean PR that supersedes #44. Will report the accept/reject split.