[Benchmark] Add support for HERBench benchmark

[GabrieleSerussi]

Description

This PR adds HERBench (paper · project page · dataset · code), a CVPR 2026 benchmark for multi-evidence integration in video question answering. Every question is a five-way MCQ that requires aggregating at least 3 distinct, temporally separated visual cues from a long video (avg. ~6.6 min, mean Minimum-Required-Frame-Set ≈ 5.5). SOTA models reach only 31–42% (random = 20%).

Datasets added (plus _8frame/_16frame/_32frame/_64frame/_1fps variants in video_dataset_config.py):

Name	HF config	Questions	Videos	Video download
HERBench	full	27,631	335	~161 GB
HERBench_lite	lite	2,000	68	~35 GB
HERBench_lite_v2	lite_v2	1,971	68	~35 GB

Implementation notes

• TYPE = 'Video-MCQ', subclasses VideoBaseDataset; TSV generated from the HF parquet with a JSON-encoded candidates column (per-letter TSV columns would NaN-parse the 16 choices whose literal text is "None" — verified on the real data; a handful of questions also have only 4 options, handled everywhere).
• Variant-aware download: the lite datasets fetch only the 4 archive chunks they need (~35 GB) via snapshot_download(allow_patterns=...) instead of the full ~171 GB.
• The videos ship as 17 split tar chunks that form two complete tar archives back to back; extraction streams straight from the chunks with tarfile(..., ignore_zeros=True) — no ~160 GB intermediate concatenated tar (halves the extra disk needed), resumable (skips already-extracted files).
• Evaluation follows the official HERBench protocol: letter extraction (ported from the official repo, restricted to the letters each question offers) + exact match, with the standard build_judge/extract_answer_from_item LLM fallback when --judge is set (default exact_matching); missing/failed predictions always score 0 and never reach the judge. Reports overall accuracy plus per-task-type (12 types) and per-source breakdowns.

Testing

• GPU end-to-end: Qwen2.5-VL-3B through the real infer_data_job_video loop + evaluate() on a HERBench_lite_v2 slice — 100% of answers parsed, accuracy in the expected range.
• Real end-to-end run of HERBench_lite and HERBench_lite_v2: download, streaming extraction of all 68 videos, TSV generation, build_prompt in both frame (decord) and video-LLM modes, evaluate() with simulated predictions (oracle predictions score exactly 1.0; per-task breakdown verified).
• Streaming extraction unit-tested against a synthetic two-tar split fixture (byte-identical output, GNU LongLink names, skip-existing on rerun).
• flake8 clean.