feat(tasks): add LongProc benchmark (6 task types, 16 configs)

[xiye17]

Summary

Hi,

This is the author from LongProc, trying to:

• Add the LongProc long-form procedural generation benchmark
  (paper, COLM 2025) as a new task suite
• Dataset: `PrincetonPli/LongProc` on HuggingFace Hub
• 6 task types, 16 configs covering output lengths from 0.5k to 8k tokens:

  Task	Configs	Primary Metric
  countdown	0.5k, 2k, 8k	accuracy
  path_traversal	0.5k, 2k, 8k	accuracy
  tom_tracking	0.5k, 2k, 8k	accuracy
  html_to_tsv	0.5k, 2k, 8k	F1
  pseudo_to_code	0.5k, 2k	accuracy
  travel_planning	2k, 8k	accuracy

• Group hierarchy: `longproc` → 6 sub-groups → 16 leaf tasks
• All tasks aggregate via a unified `score` metric

Files added (29)

• `lm_eval/tasks/longproc/utils.py` — `process_docs` to parse metadata JSON
• `lm_eval/tasks/longproc/metrics.py` — 6 evaluation functions ported from original codebase
• 3 base YAML configs (`default_yaml{0.5k,2k,8k}`)
• 16 individual task YAMLs
• 7 group YAMLs (6 sub-groups + 1 top-level)

Optional dependencies

• pandas: required only for `html_to_tsv` evaluation (guarded import)
• g++ (C++11): required only for `pseudo_to_code` evaluation (compiles and runs model-generated C++)

Test plan

• `python -c 'from lm_eval.tasks import TaskManager; tm = TaskManager(); print([t for t in tm.all_tasks if "longproc" in t])'` lists all 23 entries
• `lm_eval --model dummy --tasks longproc_countdown_0.5k --limit 50` runs without error
• `lm_eval --model dummy --tasks longproc --limit 50` tests full group aggregation
• Metric values match original LongProc evaluators on sample outputs

[CLAassistant]

All committers have signed the CLA.

[baberabb]

Hi! Thanks for the PR! LGTM, just one small addition, could you add unsafe_code: true to the configs? We use that flag to warn users when a task executes arbitrary code.

[xiye17]

Thanks a lot for your quick review! I just marked the two subtasks requiring code execution. Let me know if I need to add the flag more globally for the whole group.

[xiye17]

Hi! Thanks for the PR! LGTM, just one small addition, could you add unsafe_code: true to the configs? We use that flag to warn users when a task executes arbitrary code.

Hi I just mark the unsafe_code: true to default yaml files and consolidate the git commit logs. Thanks!