All issues

Search results

Note: reasoning/instruction models need a large generation budget or scores silently collapse

Status: Open.

#55 In HazyResearch/legalbench;

· jimdawdy-hub opened on Jun 2, 2026

HuggingFace dataset requires trust_remote_code, broken on datasets >= 3.x

Status: Open.

#54 In HazyResearch/legalbench;

· jimdawdy-hub opened on Jun 2, 2026

evaluate_successor_liability: substring class matching credits negated mentions

Status: Open.

#53 In HazyResearch/legalbench;

· jimdawdy-hub opened on Jun 2, 2026

Instructions to run LegalBench HELM Evaluation

Status: Open.

#46 In HazyResearch/legalbench;

· widmogrod opened on Jul 29, 2025

Status of LegalBench in June 2025?

Status: Open.

#43 In HazyResearch/legalbench;

· cadamsdotcom opened on Jun 14, 2025

Update helm_prompt_settings.jsonl to allow for evaluation of all tasks

Status: Open.

#41 In HazyResearch/legalbench;

· JoelNiklaus opened on Sep 24, 2024

Evaluation data of rule_qa for GPT4, GPT3.5, and Claude

Status: Open.

#36 In HazyResearch/legalbench;

· alirezamshi opened on May 15, 2024

diversity_jurisdiction tasks are missing the example COT promt

Status: Open.

#15 In HazyResearch/legalbench;

· klarakaleb opened on Mar 13, 2023