CODEWITHSUNDEEP
tensorflowkerastputensorflow-lite
Jim Chen
Jim Chen

Reputation: 3729

Convert keras model to quantized tflite lost precision

I'm trying to convert my keras model into tflite quantized model so that I can run my model on coral TPU, but the output of my keras model and tflite model are significantly different.

The red points are quantized tflite model output, and blue points are original keras model output.

enter image description here

Here is my code to convert keras model to quantized tflite model :

quant = True
gc.collect()
import tensorflow as tf
import numpy as np
import pathlib
print(tf.__version__)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
if quant:
    print("Converting quant....")
    sample_size = 200
    rdm_idx = np.random.choice(range(len(X_test)),sample_size)
    rep_data = tf.cast(X_train[rdm_idx], tf.float32) / 255.0
    dataset = tf.data.Dataset.from_tensor_slices(rep_data).batch(1)

    def representative_data_gen():
        for input_value in dataset.take(sample_size):
            yield [input_value]

    converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
    converter.representative_dataset = representative_data_gen
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
    converter.inference_input_type = tf.uint8
    converter.inference_output_type = tf.uint8
    tflite_model_quant = converter.convert()
    open("MaskedLandMarkDetction_MobileNetV2_quant_fromKeras_v5.tflite", "wb").write(tflite_model_quant)

    print("Write quantization tflite done.")
else:
    print("Converting normal....")
    tflite_model = converter.convert()
    open("MaskedLandMarkDetction_MobileNetV2_fromKeras.tflite", "wb").write(tflite_model)
    print("Write tflite done.")

X_train is my training data, and I will scale input images value from 0 to 1 by divided 255., so I do the same in representative_data_gen functions.

Any assistance you can provide would be greatly appreciated.

The tensorflow version I used is gpu 2.2.0

Upvotes: 1

Views: 893

Answers (1)

suharshs
suharshs

Reputation: 1088

It looks like the usage of the api is correct. Not all models are guaranteed to get good accuracy with post training quantization. For example, tasks that require more precision or small models may suffer more loss.

For these more difficult tasks we recommend using quantization aware training, which is available for keras models: https://www.tensorflow.org/model_optimization/guide/quantization/training.

Upvotes: 2

Related Questions