Skip to content

Adopt HSLVision (first public HSL detection benchmark) — dataset integration + 512-res retrain #2822

Description

@CodeByDamianK

Summary

Whirlwind Amsterdam (ex-Dutch Nao Team, UvA) released HSLVision — the first public object
detection dataset/benchmark for the unified Humanoid Soccer League:
https://huggingface.co/datasets/whirlwind-ams/hslvision (CC BY 4.0) ·
https://intelligentroboticslab.github.io/hslvision/
6,068 images / 65,309 boxes across 6 venues (LAB42, German Open 2026 Cologne, Beijing "Ice
Ribbon" Masters, Salvador, Sydney, Nagoya), Booster K1+T1 + other KidSize robots, splits
4,247/910/911.

Why it matters — the baseline collapses on it

Evaluating our NAO-only RF-DETR baseline on hslvision-valid (910 imgs, first real in-domain
labeled benchmark we've ever had):

benchmark overall mAP@50:95 Ball Robot GoalPost LSpot TSpot XSpot
NAO val (in-domain) 0.542 0.848 0.819 0.679 0.238 0.202 0.274
hslvision valid (HSL) 0.133 0.538 0.194 0.000 0.000 0.000 0.001

Ball transfers (consistent with the bake-off), Robot collapses (NAO→Booster appearance shift),
GoalPost/junctions hit ~0 — note the junction zeros are partly a labeling-convention mismatch
(we predict ~10px point-boxes; hslvision GT uses ~35px junction-arm boxes), so they overstate
pure blindness — but "useless under the benchmark's convention" is the operative reality.

Done

  • Downloaded (6.19 GB) + converted to our COCO layout: parquet → data/hslvision_coco/,
    class remap onto our 7-class taxonomy (their K1 class → Robot; verified visually it
    covers ALL humanoid robot types). Conversion is lossless: 6,068 imgs / 65,309 anns exact.
  • Audit: per-class counts (train: 12.5k LSpot, 9.1k TSpot, 3.5k XSpot, 10.5k Robot, 2.3k
    Ball…), box-size stats, per-class crop montages + overlays (label quality credible,
    GoalPost = per-post boxes like ours), venue/filename mapping, htwk_T1 leakage check
    (0 filename collisions; content overlap for Salvador/Beijing footage still flagged).
  • Config fixes unblocked by this: stale %56 divisibility validator → %32; resolution
    448 → 512 (RFDETRSmall native); fixed the dead-code augmentation bug (YAML aug block
    was parsed but never forwarded to model.train(aug_config=...)).
  • Reusable eval_coco.py per-class benchmark harness (self-test gate: reproduces the known
    NAO plateau 0.542 before any new number is trusted).
  • Run A kicked off: fine-tune from the NAO best checkpoint @512 on coco_mix_v1 =
    NAO train (13,106) + hslvision train ×3 oversample (12,741 entries), 183k annotations;
    early stopping tracks hslvision-valid (deployment domain). hslvision-test (911) stays
    untouched as the final holdout.

Next

  • Run A results: per-class AP on hslvision-valid + NAO-val regression check (don't lose the
    ball/robot strength) → post table here.
  • Run B ablation: hslvision-only fine-tune (quantifies whether NAO data helps or hurts).
  • Consider Person class (hslvision has none; its unlabeled humans-as-background already
    suppress the human-as-robot FPs from the bake-off).

Attribution: Xavier Catarrinho, D.M. et al., "HSLVision: A Multimodal Vision Dataset for
RoboCup Humanoid Soccer", RoboCup 2026 Symposium. CC BY 4.0.

Metadata

Metadata

Assignees

Type

No type

Projects

Status
In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions