Skip to content

Add model-validation-configs for Llama-4-Scout-17B-16E-Instruct-NVFP4#79

Open
wjhrdy wants to merge 1 commit intomainfrom
add-hpu-model-configs
Open

Add model-validation-configs for Llama-4-Scout-17B-16E-Instruct-NVFP4#79
wjhrdy wants to merge 1 commit intomainfrom
add-hpu-model-configs

Conversation

@wjhrdy
Copy link
Copy Markdown
Contributor

@wjhrdy wjhrdy commented Apr 7, 2026

Summary

  • Add accuracy and performance configs for meta-llama/Llama-4-Scout-17B-16E-Instruct-NVFP4
  • Needed for HPU/Gaudi model validation (currently commented out in nm-cicd hpu-models.txt due to missing configs)
  • Ground truth values based on base Llama-4-Scout model, pending HPU validation run to calibrate

Test plan

  • Run lm-eval with model_validation_configs_ref=add-hpu-model-configs to validate configs load correctly

Add accuracy and performance configs for HPU/Gaudi model validation.
Ground truth values based on base Llama-4-Scout model, pending HPU validation.

Signed-off-by: Willy Hardy <whardy@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant