Skip to content

tensorrt int8精度 量化模型部署失败 #4837

@duskkkrua

Description

@duskkkrua

paddlex 部署ppseglite模型训练出来,并量化出来的模型。
我设置engine_config.precision = 2;
关于bool use_calib_mode = false;和bool use_calib_mode = true;
我都试过了,都无法部署。
bool use_calib_mode = false;情况下
输出
I1212 18:30:30.523234 36012 tensorrt_subgraph_pass.cc:302] --- detect a sub-graph with 6 nodes
I1212 18:30:30.568235 36012 tensorrt_subgraph_pass.cc:846] Prepare TRT engine (Optimize model structure, Select OP kernel etc). This process may cost a lot of time.
W1212 18:30:32.035461 36012 place.cc:161] The paddle::PlaceType::kCPU/kGPU is deprecated since version 2.3, and will be removed in version 2.4! Please use Tensor::is_cpu()/is_gpu() method to determine the type of place.
I1212 18:30:32.036458 36012 engine.cc:301] Run Paddle-TRT Dynamic Shape mode.
W1212 18:30:32.036458 36012 helper.h:127] Calibrator is not being used. Users must provide dynamic range for all tensors that are not Int32 or Bool.
E1212 18:30:32.037458 36012 helper.h:131] 4: [standardEngineBuilder.cpp::nvinfer1::builder::`anonymous-namespace'::initCalibrationParams::1463] Error Code 4: Internal Error (Calibration failure occurred with no scaling factors detected. This could be due to no int8 calibrator or insufficient custom scales for network layers. Please see int8 sample to setup calibration correctly.)
E1212 18:30:32.037458 36012 helper.h:131] 2: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

于是我根据提示,设置为true。
报错在
I1212 18:32:06.368043 35968 memory_optimize_pass.cc:118] The persistable params in main graph are : 7.4912MB
I1212 18:32:06.369045 35968 memory_optimize_pass.cc:246] Cluster name : bilinear_interp_v2_6.tmp_0 size: 60
I1212 18:32:06.369045 35968 memory_optimize_pass.cc:246] Cluster name : shape_0.tmp_0_slice_1 size: 4
I1212 18:32:06.369045 35968 memory_optimize_pass.cc:246] Cluster name : shape_0.tmp_0_slice_2 size: 4
e[1me[35m--- Running analysis [ir_graph_to_program_pass]e[0m
I1212 18:32:06.386045 35968 analysis_predictor.cc:1838] ======= optimize end =======
I1212 18:32:06.387042 35968 naive_executor.cc:200] --- skip [feed], feed -> x
I1212 18:32:06.387042 35968 naive_executor.cc:200] --- skip [save_infer_model/scale_0.tmp_0], fetch -> fetch
Using optimized shape for warmup: 1x3x128x208
Starting model warmup with 5 iterations...
W1212 18:32:06.487042 35968 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 13.0, Runtime API Version: 11.8
W1212 18:32:06.495042 35968 gpu_resources.cc:164] device: 0, cuDNN Version: 8.6.
I1212 18:32:06.496042 35968 tensorrt_engine_op.h:435] This process is generating calibration table for Paddle TRT int8...
然后error
没有为 paddle_inference.dll 加载的符号文件。
但我是有这个文件,并且在fp16模式下正常运行。

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions