Tf lite fused RELU activation doesn't work

Question

I'm trying to extract the output of an int8 quantized model to debug model accuracy. I just realized that that Conv2d(relu) output contains a lot of negative values. I know that the Relu activation is fused to conv2d. Am I wrong or the tflite model has a bug?

Below is my script:

import tensorflow as tf
from tensorflow.keras.applications import ResNet50#, MobileNetV2
from tensorflow.keras.applications.resnet50 import preprocess_input
import tensorflow_datasets as tfds

resnet50 = ResNet50(
    include_top=True,
    weights='imagenet',
    input_tensor=None,
    input_shape=(224, 224, 3),
    pooling=None,
    classes=1000,
    classifier_activation='softmax'
)
resnet50.summary()

imagenet_v2 = tfds.load('imagenet_v2', split='test', as_supervised=True, shuffle_files=True,
                   download_and_prepare_kwargs={
                       'download_config': tfds.download.DownloadConfig(verify_ssl = False)
                   })

def representative_dataset():
    dataset = imagenet_v2.shuffle(1000).batch(1).take(200)
    for data, _ in dataset:
        image = tf.image.resize(data, [224, 224])
        yield [image]


converter = tf.lite.TFLiteConverter.from_keras_model(resnet50)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS_INT8,
    tf.lite.OpsSet.SELECT_TF_OPS
]
converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.int8  # or tf.uint8
converter.inference_output_type = tf.int8  # or tf.uint8
uint8_quantized_model = converter.convert()

interpreter = tf.lite.Interpreter(model_content=uint8_quantized_model, experimental_preserve_all_tensors=True)
interpreter.allocate_tensors()
dataset = imagenet_v2.shuffle(1000).batch(1).take(10)

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

for img, label in dataset:
    img = tf.image.resize(img, (224, 224))
    img = preprocess_input(img)
    img = tf.cast(img, tf.int8)

    interpreter.set_tensor(input_details[0]['index'], img)
    interpreter.invoke()
    output = interpreter.get_tensor(output_details[0]['index'])

    print(interpreter.get_tensor(113).flatten())

This is the visualized model:

And this is my output:

I have tried to change the quantized options. The result is still the same.

Answer 1

The values you posted are the quantized output. In the image with your model visualization, notice that the equation for the dequantization is 0.1334 * (q + 128). This means that 128 will be added to the quantized values before being multiplied by 0.1334. Since -128 is the minimum value of an int8, the dequantized values will all be non-negative.