Skip to content

feat: add TimeLens benchmark#1323

Merged
kcz358 merged 1 commit into
mainfrom
feat/timelens
May 6, 2026
Merged

feat: add TimeLens benchmark#1323
kcz358 merged 1 commit into
mainfrom
feat/timelens

Conversation

@kcz358
Copy link
Copy Markdown
Collaborator

@kcz358 kcz358 commented May 6, 2026

Summary

  • Adds TimeLens temporal grounding tasks for ActivityNet, Charades, and QVHighlights.
  • Uses kcz358/timelens with lmms-eval's existing dataset video zip extraction flow.
  • Reports Charades-STA IOU@3, IOU@5, IOU@7, and mIOU directly in lmms-eval results.

Dataset Assets

  • Uploaded 18 video zip archives to kcz358/timelens under videos/.
  • Verified every archive is below 5GB and preserves relative paths such as charades/<video>.mp4.

Testing

  • pre-commit run --files lmms_eval/tasks/timelens/timelens.yaml lmms_eval/tasks/timelens/_default_yaml_template lmms_eval/tasks/timelens/timelens_activitynet.yaml lmms_eval/tasks/timelens/timelens_charades.yaml lmms_eval/tasks/timelens/timelens_qvhighlights.yaml lmms_eval/tasks/timelens/utils.py lmms_eval/tasks/charades_sta/charades.yaml lmms_eval/tasks/charades_sta/utils.py
  • python -m compileall -q lmms_eval/tasks/timelens lmms_eval/tasks/charades_sta
  • scripts/run_timelens_qwen3vl4b.sh with LIMIT=4
  • Focused Charades-STA parsing and direct metric aggregation smoke checks.

@kcz358 kcz358 merged commit 26f1d8e into main May 6, 2026
1 of 2 checks passed
@kcz358 kcz358 deleted the feat/timelens branch May 6, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant