CODEWITHSUNDEEP
pythonkerasresnet
Michał Gacka
Michał Gacka

Reputation: 3091

Image preprocessing as a Keras lambda layer vs preprocessing on its own

I'm trying to apply preprocessing imported from a resnet50 module in Keras before propagating the images through the network. Since I got errors when trying to apply it in the generator object, I added it to the network as a lambda layer.

But since the model was worse than one that I trained previously when I was applying preprocessing on each image separately, I compared the results of both approaches and they seem very different, although I can't see what's different about the applied operations.

import keras
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input

model = keras.models.Sequential()
model.add(keras.layers.Lambda(preprocess_input, name='preprocessing', input_shape=(224, 224, 3)))

file = '/path/to/an/image.jpeg'
x = np.array(image.img_to_array(image.load_img(file, target_size=(224, 224))))

preprocessed_x = preprocess_input(x)
predicted_x = model.predict(x.reshape(1,224,224,3)).reshape(224,224,3)

Results (left: preprocessing function used directly; right: preprocessing as a lambda layer.

Using the preprocessing function directly Using preprocessing function in a model as a lambda function

Upvotes: 2

Views: 1388

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86620

I think this may be a bug in keras code, but I found that preprocess_input(x) is modifying x in place.

See:

x = np.array(image.img_to_array(image.load_img(file, target_size=(224, 224))))
print("x before preprocess - max: " + str(x.max()) + " / min: " + str(x.min()))
preprocessed_x = preprocess_input(x)
print("x after preprocess - max: " + str(x.max()) + " / min: " + str(x.min()))

Results:

x before preprocess - max: 255.0 / min: 0.0
x after preprocess - max: 151.061 / min: -123.68

I'd certainly not expect this behavior at first, but thinking twice, it spares memory (which is valuable when neural networking...).

If you run your test with the model version before the numpy version, the problem will not happen (because the model, working in tensors, won't modify x in place).

Preprocess within or outside the model?

I'd say it's better to preprocess images outside the model, in the generator. This way you get a faster execution when training and predicting.

Upvotes: 4

Related Questions