Keras error: "If your data is in the form of symbolic tensors, you should specify the `steps` argument (instead of the `batch_size` argument.....)

Question

I want to use a custom loss function, by extracting features from the penultimate layer of VGG16 network. So, in the model.compile() function, I pass a function vgg_loss() which returns the required loss. It is shown below:

model_loss = VGG16(include_top=False, input_shape=input_size)
model.compile(optimizer = Adam(lr = lrate, decay = 1e-6), loss = vgg_loss(model_loss))

The vgg_loss() function is defined as follows:

import keras.backend as K
from keras.backend import reshape
from keras.applications.vgg16 import preprocess_input

def vgg_loss(model):

    def loss(gt, pred):
        print(" Inside loss function ")    

        pred = reshape(pred, (1,) + (128,128,3))
        gt = reshape(gt, (1,) + (128,128,3))

        gt_in = preprocess_input(gt)
        pred_in = preprocess_input(pred)

        pred_out = model.predict(pred_in)
        gt_out = model.predict(gt_in)

        return K.mean(K.mean((pred_out - gt_out)**2, axis = (1,2)), axis = 1)

    return loss

When run model.compile, I get the following error related to the usage of symbolic tensors:

Traceback (most recent call last):

File "C:\Users\Dell\.spyder-py3\UNet_keras\train_SO.py", line 32, in <module>
model = unet(input_size = (height,width,nc))

File "C:\Users\Dell\.spyder-py3\UNet_keras\model_SO.py", line 68, in unet
loss = vgg_loss(model_loss), metrics = ['mse'])

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training.py", line 229, in 
compile
self.total_loss = self._prepare_total_loss(masks)

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training.py", line 692, in 
_prepare_total_loss
y_true, y_pred, sample_weight=sample_weight)

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\losses.py", line 71, in __call__
losses = self.call(y_true, y_pred)

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\losses.py", line 132, in call
return self.fn(y_true, y_pred, **self._fn_kwargs)

File "C:\Users\Dell\.spyder-py3\UNet_keras\data_SO.py", line 28, in loss
pred_out = model.predict(pred)

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training.py", line 1464, 
in predict
callbacks=callbacks)

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training_arrays.py", line 
250, in predict_loop
steps_name='steps')

File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training_utils.py", line 
571, in check_num_samples
'you should specify the `' + steps_name + '` argument '

ValueError: If your data is in the form of symbolic tensors, you should specify the `steps` argument 
(instead of the `batch_size` argument, because symbolic tensors are expected to produce batches of 
input data).

What am I doing wrong here?

Answer 1

1. You have to use tensorflow-2 instead of keras.

2. You can't add anything in the loss function and expect it to work, it must be differentiable. You may not need to add preprocessing_input of vgg16 inside the loss, your output from the model shouldn't be perfect images (with range 0-255), they should be already normalized tensors (due to some activations like sigmoid). (Make sure, the VGG16 you're using was also trained with same normalization scheme (0-1) range, as the output from the network will probably pass through an activation function like sigmoid, your output will most likely be in the range(0-1))

Here's a simple model with the loss function of yours:

import tensorflow.keras.backend as K
from tensorflow.keras.applications import *
from tensorflow.keras.layers import *
from tensorflow.keras.models import *

def vgg_loss(model):

    def loss(y_true, y_pred):
        return K.mean(K.square(model(y_pred) - model(y_true)), axis=-1)
    return loss

input_size = (224,224,3)
model_loss = VGG16(include_top=False, input_shape=input_size)

# simple model

ip = Input(input_size)
base_model = Dense(3)(ip)

model = Model(ip, base_model)

model.summary()

model.compile(optimizer = 'adam', loss = vgg_loss(model_loss))

Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         [(None, 224, 224, 3)]     0         
_________________________________________________________________
dense (Dense)                (None, 224, 224, 3)       12        
=================================================================
Total params: 12
Trainable params: 12
Non-trainable params: 0