Description
Among the 66 models running end-to-end in TT-NN, the following models exhibit accuracy deviations:
Models with Accuracy Issues:
- Bloom: 43.3
- albert/albert-base-v2: 69.04
- albert/albert-large-v2: 23.63
- albert/albert-xlarge-v2: 53.42
- mobilenetv1_100.ra4_e3600_r224_in1k: -6.46
- regnet_x_16gf: 1.01
- regnet_x_32gf: 3.94
Models with Accuracy Marked as N/A:
- Unet-brain: N/A
- ssd300_vgg16: N/A
Action Required:
- Investigate and diagnose the causes of the accuracy deviations.
- If the issue is in TT-NN operation, determine what operation on what input causes an issue. Fire a ticket to tt-metal, link here
- Validate expected accuracy baselines for each model.
- Assess Unet-brain to determine if accuracy reporting is feasible.
Sub-issues
Metadata
Metadata
Assignees
Type
Projects
Status
In Progress