Skip to content

Pull requests: open-compass/VLMEvalKit

New pull request New
81 Open 989 Closed
81 Open 989 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[Model] Support Qwen3.8 Series
#1671 opened Sep 4, 2026 by RishabhPatil Loading…
1
fix(inference): unpack structured records per sample, not all-or-nothing
#1669 opened Sep 3, 2026 by Anai-Guo Contributor Loading…
1
fix(molrecbench_wild): give the converter its required track column
#1668 opened Sep 3, 2026 by Anai-Guo Contributor Loading…
[Benchmark] Add support for HERBench benchmark
#1666 opened Sep 1, 2026 by GabrieleSerussi Loading…
Add LFM2.5-VL-3B benchmark handling
#1663 opened Aug 28, 2026 by rshube Loading…
3
[Model] update bluelm model api
#1659 opened Aug 27, 2026 by tanhui2975 Loading…
[Fix] Molmo AI2D: map no-letter answers to the option letter, not its…
#1655 opened Aug 24, 2026 by aarjunsrinivasan Loading…
fix(dataset): preserve literal 'None' options in TSV loading and MCQ prompt building
#1651 opened Aug 23, 2026 by Xayar145 Loading…
fix(distributed): default LOCAL_RANK to 0 instead of 1 in run.py
#1650 opened Aug 23, 2026 by Xayar145 Loading…
[Fix]Fix CCOCR evaluation for long text inputs
#1647 opened Aug 20, 2026 by XiaoBao-175 Loading…
[Benchmark] Add ActiveVision benchmark
#1646 opened Aug 20, 2026 by lzpp2598 Contributor Loading…
[Fix] Keep Thyme sandbox images in user observation turns
#1645 opened Aug 18, 2026 by reacher-z Loading…
1
[Feature] Add multi-turn chat for Llama 4, Llama Vision, and Qwen3-VL
#1644 opened Aug 18, 2026 by reacher-z Loading…
[Fix] Detect MIME types for extensionless images
#1643 opened Aug 18, 2026 by reacher-z Loading…
1
[Fix] Evaluate MMMU open-ended answers by content
#1642 opened Aug 18, 2026 by reacher-z Loading…
[Fix] Accept numeric LogicVista judge answers
#1641 opened Aug 18, 2026 by reacher-z Loading…
[Benchmark] Upgrade OmniDocBench to v1.5
#1640 opened Aug 18, 2026 by reacher-z Loading…
1
[Dataset] Add downloadable OmniMat data URLs
#1639 opened Aug 18, 2026 by yansumm Loading…
[Feature] Add multi-turn chat for Gemma 3, Mantis, and Pixtral
#1637 opened Aug 16, 2026 by reacher-z Loading…
Add Japanese Heron-Bench support
#1636 opened Aug 16, 2026 by reacher-z Loading…
1
1
[Feature] Add multi-turn chat for Kimi-VL, Aria, and Phi-4
#1635 opened Aug 16, 2026 by reacher-z Loading…
[Feature] Support evaluating selected dataset indices
#1634 opened Aug 16, 2026 by reacher-z Loading…
1
[Benchmark] Add support for CAPEval
#1633 opened Aug 14, 2026 by liuzhipenggg Loading…
2
[Model] Add VQToken for LLaVA-OneVision
#1625 opened Aug 11, 2026 by Hai-chao-Zhang Loading…
1
Add C4 Bench evaluation
#1624 opened Aug 10, 2026 by sci-m-wang Loading…
2
ProTip! no:milestone will show everything without a milestone.