Skip to content

[eager_fail_to_run] resnet50_quantized_qat #2569

Open
@IvanKobzarev

Description

Running from pytorch main

python benchmarks/dynamo/torchbench.py --only resnet50_quantized_qat --accuracy --no-translation-validation --training --amp --backend inductor --disable-cudagraphs --device cuda --output out.csv
cuda train resnet50_quantized_qat
Traceback (most recent call last):
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 2744, in validate_model
    self.model_iter_fn(model, example_inputs)
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/torchbench.py", line 453, in forward_and_backward_pass
    pred = mod(*cloned_inputs)
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 824, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 400, in __call__
    raise e
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 387, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1760, in _call_impl
    return forward_call(*args, **kwargs)
  File "<eval_with_key>.3", line 167, in forward
    activation_post_process_73 = self.activation_post_process_73(fc);  fc = None
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1760, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/ao/quantization/fake_quantize.py", line 408, in forward
    return torch.fused_moving_avg_obs_fake_quant(
RuntimeError: expected scalar type Float but found Half

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 4857, in run
    ) = runner.load_model(
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/torchbench.py", line 372, in load_model
    self.validate_model(model, example_inputs)
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 2746, in validate_model
    raise RuntimeError("Eager run failed") from e
RuntimeError: Eager run failed

eager_fail_to_run

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions