feat: add extended evaluation support using elutherai eval-harness and improved TRL training by erfanMhi · Pull Request #56 · one-covenant/grail

refactor(environments): change math source to EleutherAI huggingface repo!

erfanMhi

committed

feat(evaluation): add eval_math_harness for evaluating models on Hendrycks MATH dataset

erfanMhi

committed

chore(config): change evaluation split from 'test' to 'val' for math dataset

erfanMhi

committed

feat(training): add unified TRL GRPO training script for GSM8K and MATH datasets with detailed README

erfanMhi

committed

chore(config): update hyperparameters in TRL training script for GSM8K, including learning rate, batch size, and max sequence length

erfanMhi

committed

feat(evaluation): add eval_pass_at_k script for computing pass@k metrics on MATH dataset on the test set

erfanMhi

committed

feat(cli): add parallel mining functionality with multi-GPU support and result aggregation

erfanMhi

committed

feat(evaluation): enhance eval_math_harness with vLLM backend support and tensor parallelism features

erfanMhi

committed

feat(dependencies): add lm-eval for evaluation harness with vLLM support

erfanMhi

committed

feat(cli): increase default batch size to 16 for optimal performance and add critical checks for time management in parallel mining

erfanMhi

committed

fix(cli): update nonce calculation in package_rollout_data to prevent collisions by using ROLLOUTS_PER_PROBLEM as multiplier

erfanMhi

committed

feat(model): add support for PyTorch SDPA in model loading with priority handling for Flash Attention 2

erfanMhi

committed

feat(evaluation): add AIME 2024, AMC 2023, GSM8K eval tasks for lm-harness that support reasoning model evaluation as well.

erfanMhi

committed

feat(training): enhance TRL GRPO training script with detailed hyperparameter configuration, add TrainingPassAtKTracker for pass@k metrics logging, and update evaluation callback for improved WandB…

erfanMhi

committed

feat(evaluation): introduce shared utilities for model evaluation tasks, including answer extraction and string normalization functions.

erfanMhi

committed

feat: connect importance_sampling_level to GRPOConfig

erfanMhi

committed

chore: resolve minor ci linting issues

erfanMhi

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add extended evaluation support using elutherai eval-harness and improved TRL training#56

feat: add extended evaluation support using elutherai eval-harness and improved TRL training#56
erfanMhi wants to merge 17 commits intomainfrom
research/grail-v0-blog

Commits on Dec 24, 2025

refactor(environments): change math source to EleutherAI huggingface repo!

feat(evaluation): add eval_math_harness for evaluating models on Hendrycks MATH dataset

chore(config): change evaluation split from 'test' to 'val' for math dataset

feat(training): add unified TRL GRPO training script for GSM8K and MATH datasets with detailed README

chore(config): update hyperparameters in TRL training script for GSM8K, including learning rate, batch size, and max sequence length

feat(evaluation): add eval_pass_at_k script for computing pass@k metrics on MATH dataset on the test set

feat(cli): add parallel mining functionality with multi-GPU support and result aggregation

feat(evaluation): enhance eval_math_harness with vLLM backend support and tensor parallelism features

feat(dependencies): add lm-eval for evaluation harness with vLLM support

feat(cli): increase default batch size to 16 for optimal performance and add critical checks for time management in parallel mining

fix(cli): update nonce calculation in package_rollout_data to prevent collisions by using ROLLOUTS_PER_PROBLEM as multiplier

feat(model): add support for PyTorch SDPA in model loading with priority handling for Flash Attention 2

feat(evaluation): add AIME 2024, AMC 2023, GSM8K eval tasks for lm-harness that support reasoning model evaluation as well.

feat(training): enhance TRL GRPO training script with detailed hyperparameter configuration, add TrainingPassAtKTracker for pass@k metrics logging, and update evaluation callback for improved WandB…

feat(evaluation): introduce shared utilities for model evaluation tasks, including answer extraction and string normalization functions.

feat: connect importance_sampling_level to GRPOConfig

chore: resolve minor ci linting issues

feat: add extended evaluation support using elutherai eval-harness and improved TRL training#56erfanMhi wants to merge 17 commits intomainone-covenant/grail:mainfrom research/grail-v0-blogone-covenant/grail:research/grail-v0-blogCopy head branch name to clipboard

Commits

Commits on Dec 24, 2025

feat: add extended evaluation support using elutherai eval-harness and improved TRL training#56
erfanMhi wants to merge 17 commits intomainfrom
research/grail-v0-blog