CODEWITHSUNDEEP
pythondeep-learningquantizationonnxonnxruntime
Parag Jain
Parag Jain

Reputation: 662

Quantization of Onnx model

I am trying to quantize an ONNX model using the onnxruntime quantization tool.

My code is below for quantization:

import onnx
from quantize import quantize, QuantizationMode

# Load the onnx model     
model = onnx.load('3ddfa_optimized_withoutflatten.onnx')

# Quantize
quantized_model = quantize(model, quantization_mode=QuantizationMode.IntegerOps)
 
# Save the quantized model
onnx.save(quantized_model, 'quantized_model.onnx')

After this method the model I am getting has 0 dimensional model. What arguments do I have to pass in quantize function so I can get a proper model?

Upvotes: 3

Views: 15107

Answers (2)

Shrinidhi M
Shrinidhi M

Reputation: 139

Try this:

def quantize_onnx_model(onnx_model_path, quantized_model_path):
    from onnxruntime.quantization import quantize_dynamic, QuantType
    import onnx
    onnx_opt_model = onnx.load(onnx_model_path)
    quantize_dynamic(onnx_model_path,
                     quantized_model_path,
                     weight_type=QuantType.QInt8)

    print(f"quantized model saved to:{quantized_model_path}")

quantize_onnx_model("/content/drive/MyDrive/Datasets/model.onnx", "/content/drive/MyDrive/Datasets/model_quant.onnx")

Upvotes: 2

Tianlei Wu
Tianlei Wu

Reputation: 21

Unless you share the onnx model, it is hard to tell the cause.

For OnnxRuntime 1.4.0, you can try the following:

    quantized_model = quantize(onnx_opt_model,
                               quantization_mode=QuantizationMode.IntegerOps,
                               symmetric_weight=True,
                               force_fusions=True)

If the problem still exits, please share your onnx model so that we can take a look.

Upvotes: 2

Related Questions