Trace model process will introduce extra aten::Int
op
#1185
Replies: 4 comments 1 reply
-
@narendasan @peri044 When you have time please take a look, Thanks. |
Beta Was this translation helpful? Give feedback.
-
watching |
Beta Was this translation helpful? Give feedback.
-
In my experience |
Beta Was this translation helpful? Give feedback.
-
I find a blog about tracing process. It mentioned that expressions like |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Background: When i tried to convert Pytorch model Swin-Transformer to torchscript, it didn't work with script and seemed like some ops are not supported in script process. So i have to trace the model. But during tracing process, i find it will introduce extra
aten::Int
op which isn't supported well during TorchTRT converting in some cases. In the Swin-Transformer case, i have to change the source code to avoid tracing process introduce extraaten::Int
. I wonder if we have some methods to avoid tracing introducingaten::Int
into the torchscript. There is an example about the case below.There is a snippet code which is widely used in swin-transformer. When i traced the model and fully compiled it with torch-trt, it failed during compiling. When i scripted the model and compiled it with torch-trt, it worked.
The script process log is here:
script_log.txt.
The trace process log is here:
trace_full_compile.txt
In script process,
self.window_size
is a int value%self.window_size : int = prim::Constant[value=7]()
and in trace process,self.window_size
is a Tensor value%16 : Long(requires_grad=0, device=cpu) = prim::Constant[value={7}]()
. In my opinion, this introducedaten::Int
op into the graph which leaded to the trace process failure.So i wonder if we have some methods to avoid parsing
self.window_size
as a Tensor value during trace process. I think it will avoid lots ofaten::Int
op in some cases.Beta Was this translation helpful? Give feedback.
All reactions