Why
We cannot claim "the new search is better" without ground truth. This harness is the foundation for every later ranking change (Typesense tuning, hybrid retrieval). Do this early — it is the most-neglected and most-fatal gap.
Tasks
Deliverable
Versioned eval dataset + metric scripts → Table 1 and the Evaluation section of the paper.
Why
We cannot claim "the new search is better" without ground truth. This harness is the foundation for every later ranking change (Typesense tuning, hybrid retrieval). Do this early — it is the most-neglected and most-fatal gap.
Tasks
Assemble a query test set (representative search intents: functional e.g. GFP, property e.g. strong constitutive promoter, exact-part e.g. pET, ontology terms, etc.).
Build candidate pools per query by merging top-K from old ES + new Typesense + other baselines (pooling), deduped — to avoid pool bias.
Define gold relevance format (graded 0–3 for NDCG; binary for P@k/MRR). Hand candidate pools to the PhD colleague for judging (see Issue 6).
Implement metric scripts: precision@k, MRR, NDCG (retrieval quality) + latency/throughput (efficiency).
Seed the set with known-good labels we already have, e.g. rbs → BBa_B0034, GFP → BBa_E0040.
Deliverable
Versioned eval dataset + metric scripts → Table 1 and the Evaluation section of the paper.