CODEWITHSUNDEEP
tensorflowkerastensorflow-litequantizationgoogle-coral
anilsathyan7
anilsathyan7

Reputation: 1833

Unable to properly convert tf.keras model to quantized format for coral TPU

I'am trying to convert a tf.keras model based on mobilenetv2 with transpose convolution using latest tf-nighlty. Here is the conversion code

#saved_model_dir='/content/ksaved'  # tried from saved model also
#converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)

converter = tf.lite.TFLiteConverter.from_keras_model(reshape_model)
converter.experimental_new_converter=True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8

converter.representative_dataset = representative_dataset_gen

tflite_quant_modell = converter.convert()
open("/content/model_quant.tflite", "wb").write(tflite_quant_modell)

The conversion was successful(in google colab); but it has quantize and dequantize operators at the ends(as seen using netron). All operators seems to be supported. Representative data set images are float32 in generator and the model has a 4 channel float32 input by default. It looks like we need a UINT8 input and output inside model for coral TPU. How can we properly carry out this conversion?

Ref:-

  1. Full integer quantization of weights and activations

  2. How to quantize inputs and outputs of optimized tflite model

  3. Coral Edge TPU Compiler cannot convert tflite model: Model not quantized

I tried with 'tf.compat.v1.lite.TFLiteConverter.from_keras_model_file' instead of v2 version.I got error: "Quantization not yet supported for op: TRANSPOSE_CONV" while trying to quantize the model in latest tf 1.15 (using representative dataset) and "Internal compiler error. Aborting! " from coral tpu compiler using tf2.0 quantized tflite

Tflite model @ https://github.com/tensorflow/tensorflow/issues/31368

enter image description here

It seems to work until the last constitutional block (1x7x7x160) The compiler error(Aborting) does not give any information regarding the potential cause and all types of convolutional layers seems to be supported as per coral docs.

Coral doc: https://coral.ai/docs/edgetpu/models-intro/#quantization

Upvotes: 0

Views: 1755

Answers (2)

Alexander
Alexander

Reputation: 1502

I always make sure not to use the experimental converter:

converter.experimental_new_converter = False

Upvotes: 0

Nam Vu
Nam Vu

Reputation: 1757

Here is a dummy model example of quantizing a keras model. Notice I'm using strict tf1.15 for the example, because tf2.0 deprecated:

converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8

with the from_keras_model api. I think the most confusing thing about this is that you can still call it but nothing happens. This means that model will still take in float inputs. I notice that you are using tf2.0, because from_keras_model is a tf2.0 api. Coral still suggest using tf1.15 for converting model for now. I suggest downgrading tensorflow or maybe even just use this (while keeping tf2.0, it may or may not work):

tf.compat.v1.lite.TFLiteConverter.from_keras_model_file

More on it here.

Upvotes: 1

Related Questions