Checklist
Bug Description
After pip install lmms-eval, every evaluation run fails during task registration. TaskManager.initialize_tasks() scans all task directories at startup; when it reaches screenspot it crashes:
FileNotFoundError: [Errno 2] No such file or directory:
'.../site-packages/lmms_eval/tasks/screenspot/_default_template_rec_yaml'
Because the scan is global, this kills runs for completely unrelated tasks too (e.g. mmmu_val) — you can't evaluate anything from a pip-installed lmms-eval.
Steps to Reproduce
pip install lmms-eval==0.7.1
lmms-eval --tasks list
Error Message / Traceback
FileNotFoundError: [Errno 2] No such file or directory:
'.../site-packages/lmms_eval/tasks/screenspot/_default_template_rec_yaml'
Environment
- lmms-eval 0.7.1, installed from the PyPI wheel
-
Additional Context
No response
Checklist
Bug Description
After pip install lmms-eval, every evaluation run fails during task registration. TaskManager.initialize_tasks() scans all task directories at startup; when it reaches screenspot it crashes:
FileNotFoundError: [Errno 2] No such file or directory:
'.../site-packages/lmms_eval/tasks/screenspot/_default_template_rec_yaml'
Because the scan is global, this kills runs for completely unrelated tasks too (e.g. mmmu_val) — you can't evaluate anything from a pip-installed lmms-eval.
Steps to Reproduce
Error Message / Traceback
FileNotFoundError: [Errno 2] No such file or directory: '.../site-packages/lmms_eval/tasks/screenspot/_default_template_rec_yaml'Environment
Additional Context
No response