Will int8 PTQ reduce VRAM for VGG-19? #1100
Unanswered
jonahclarsen
asked this question in
Q&A
Replies: 1 comment 1 reply
-
a pytorch nn.module (with FP32) weights vs INT8 TRT engine embedded in a torchscript module - The latter would consume less memory. However, I don't know if the memory savings would be 3-4x. You can try running a python example and check. (For reference: https://github.com/pytorch/TensorRT/blob/master/tests/py/test_ptq_dataloader_calibrator.py) |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi all,
I primarily want to use Torch-TensorRT to make VGG-19 take up 3-4x less space in VRAM than in plain Libtorch full-precision, with int8 PTQ. I am working on testing it myself but haven't yet been able to get PTQ working (#1091).
Does anyone have experience using PTQ with VGG who can comment on if VGG-19 will use significantly less VRAM after int8 PTQ?
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions