CODEWITHSUNDEEP
pytorchtensorrt
yy G
yy G

Reputation: 5

when I use the trtexec --onnx=** --saveEngine=** to transfer my onnx to a trt model, I got a warning from TensorRT

My workflow is like: pytorch --> onnx --> trt. I use torch.onnx.export() function to export my model with a FP16 precision. And then I use the trtexec --onnx=** --saveEngine=** to transfer my onnx file to a trt model,a warning came out like: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. I don't know why and does anyone who can help me with this?

I tried what I can to solve this, but seemly noting can work.

Upvotes: 0

Views: 5537

Answers (2)

Erick Cardozo Gálvez
Erick Cardozo Gálvez

Reputation: 16

Inference with explicit batch onnx model, try:

trtexec --explicitBatch --onnx=your_model.onnx

https://elinux.org/TensorRT/ONNX#How_to_use_trtexec_to_run_inference_with_dynamic_shape.3F

Upvotes: 0

R1608
R1608

Reputation: 23

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL There is limited support for INT64. TensorRT will attempt to cast down INT64 to INT32 clamping values to +-INT_MAX if necessary

follow this link for the the support matrix of ONNX operators in ONNX-TensorRT https://github.com/onnx/onnx-tensorrt/blob/main/docs/operators.md

Upvotes: 0

Related Questions