CODEWITHSUNDEEP
pythontensorflowtensorflow2.0convolutionquantization
Kalen White
Kalen White

Reputation: 69

TensorFlow 2.0 TFLite Quantized export with UINT8 weights

In the previous TensorFlow version I used (1.9) I was able to quantize my network with UINT8 weights stored in the Conv2D operation inputs. Now with TensorFlow 2.0 using a Keras model, post-training quantization gives me INT8 weights with seemingly no option for unsigned weights. Is it not possible to control the sign on the weights in Conv layers with TF 2.0?

import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [input]
converter.representative_dataset = representative_dataset_gen
# there is no such tf.lite.OpsSet.TFLITE_BUILTINS_UINT8
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8  # or tf.int8 ( note this has zero effect on the tensors produced for Conv2D operations -- all of which include signed int8 unless you were to output the model as float16/32 )
converter.inference_output_type = tf.uint8  # or tf.int8
tflite_quant_model = converter.convert()

Upvotes: 2

Views: 1490

Answers (1)

Kalen White
Kalen White

Reputation: 69

The most recent version of TensorFlow (2.5) utilizes a more robust quantization scheme on the Convolutional networks where each filter depth maps to different quantization levels. As of current there is not a way to utilize the prior method.

Upvotes: 1

Related Questions