Skip to content

Conversation

@libohao1201
Copy link
Contributor

Add dynamo model list for different test scopes.

Copilot AI review requested due to automatic review settings January 22, 2026 15:34
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Dynamo model lists for different benchmark test scopes (p0, p1, p2) across three model suites: TorchBench, TIMM, and HuggingFace. The lists are organized by priority/scope level to support different testing configurations.

Changes:

  • Added TorchBench model lists for training and inference modes in p2 scope
  • Added TIMM and HuggingFace model lists for p0, p1, and p2 scopes
  • Created structured text files with numbered model entries for benchmark testing

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
.ci/benchmarks/p2/torchbench_models_list_train.txt 28 TorchBench models for training scope in p2 tests
.ci/benchmarks/p2/torchbench_models_list_infer.txt 35 TorchBench models for inference scope in p2 tests
.ci/benchmarks/p2/timm_models_list.txt 27 TIMM models with batch sizes for p2 scope
.ci/benchmarks/p2/huggingface_models_list.txt 6 HuggingFace models with batch sizes for p2 scope
.ci/benchmarks/p1/torchbench_models_list.txt 29 TorchBench models for p1 scope
.ci/benchmarks/p1/timm_models_list.txt 20 TIMM models with batch sizes for p1 scope
.ci/benchmarks/p1/huggingface_models_list.txt 20 HuggingFace models with batch sizes for p1 scope
.ci/benchmarks/p0/torchbench_models_list.txt 16 TorchBench models for p0 scope
.ci/benchmarks/p0/timm_models_list.txt 16 TIMM models with batch sizes for p0 scope
.ci/benchmarks/p0/huggingface_models_list.txt 16 HuggingFace models with batch sizes for p0 scope

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mobilenetv3_large_100 512
poolformer_m36 128
regnety_002 1024
res2net101_26w_4s
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model entry 'res2net101_26w_4s' is missing a batch size parameter, while all other entries in the file include one (e.g., '128', '512'). For consistency and to prevent potential parsing errors, add a batch size value.

Suggested change
res2net101_26w_4s
res2net101_26w_4s 128

Copilot uses AI. Check for mistakes.
@github-actions
Copy link

Performance outliers, please check!

  • 🟡 [80%, 90%), may be fluctuations
Category Model Target vs. Baseline [Eager] Target vs. Baseline [Inductor]
torchbench_bfloat16_training resnet18 0.94539 0.815009

@github-actions
Copy link

Performance outliers, please check!

  • 🔴 [-1, 80%), should be regression
Category Model Target vs. Baseline [Eager] Target vs. Baseline [Inductor]
torchbench_bfloat16_training resnet18 0.92947 0.765747
  • 🟡 [80%, 90%), may be fluctuations
Category Model Target vs. Baseline [Eager] Target vs. Baseline [Inductor]
torchbench_bfloat16_training mnasnet1_0 1.011198 0.825302

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants