feat: add JumpScore evaluation task#1329

Merged

kcz358 merged 11 commits into

EvolvingLMMs-Lab:mainfrom

mathCrazyy:main

May 11, 2026

Commits on May 11, 2026

feat: add jump rope evaluation task
mathCrazyy
committed
fix(mmmu): lazy-load judge server to avoid OpenAI API key error on module import
mathCrazyy
committed
Revert "fix(mmmu): lazy-load judge server to avoid OpenAI API key error on module import"
mathCrazyy
committed
fix(jump_rope): lazy-load HF dataset snapshot to avoid import-time download
mathCrazyy
committed
fix(mmmu): lazy-load judge server to avoid OpenAI API key error on module import
mathCrazyy
committed
ci(task-input-ab): gracefully skip comparison when BASE snapshot fails
mathCrazyy
committed
refactor(jump_rope): rename task directory from jump_rope to jumpscore
mathCrazyy
committed
Revert "fix(mmmu): lazy-load judge server to avoid OpenAI API key error on module import"
mathCrazyy
committed
Revert "ci(task-input-ab): gracefully skip comparison when BASE snapshot fails"
mathCrazyy
committed
fix(jumpscore): configure video cache in yaml
mathCrazyy
committed
fix(jumpscore): expose map metric
mathCrazyy
committed