Keras Output tensors to a Model must be the output of a Keras `Layer` (thus holding past layer metadata)

Question

I am trying to implement unpooling masks in Keras. I have a VGG encoder that outputs a specific feature map like relu5_1 and a list of unpooling masks.

def VGG19(input_tensor=None, input_shape=None, target_layer=1):
    """
    VGG19, up to the target layer (1 for relu1_1, 2 for relu2_1, etc.)
    """
    if input_tensor is None:
        inputs = Input(shape=input_shape)
    else:
        inputs = Input(tensor=input_tensor, shape=input_shape)

    layer, unpooling_masks = vgg_layers(inputs, target_layer)

    model = Model(inputs, [layer, unpooling_masks], name='vgg19')
    load_weights(model)
    return model, unpooling_masks


def vgg_layers(inputs, target_layer):
    unpooling_masks = []
    # Block 1
    x_b1 = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1')(inputs)    
    x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2')(x_b1)
    before_pooling = x     
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)
    unpooling_masks.append(make_unpooling_mask(x, before_pooling))

    # Block 2
    x_b2 = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1')(x)
    x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2')(x_b2)
    before_pooling = x 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x)
    unpooling_masks.append(make_unpooling_mask(x, before_pooling))

    # Block 3
    x_b3 = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1')(x)

    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2')(x_b3)
    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3')(x)
    x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv4')(x)
    before_pooling = x 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool')(x)
    unpooling_masks.append(make_unpooling_mask(x, before_pooling))

    # Block 4
    x_b4 = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1')(x)

    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2')(x_b4)
    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3')(x)
    x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv4')(x)
    before_pooling = x 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x)
    unpooling_masks.append(make_unpooling_mask(x, before_pooling))

    # Block 5
    x_b5 = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1')(x)

    if target_layer == 5:
        return x_b5, unpooling_masks
    elif target_layer == 4:
        return x_b4, unpooling_masks
    elif target_layer == 3:
        return x_b3, unpooling_masks
    elif target_layer == 2:
        return x_b2, unpooling_masks
    elif target_layer == 1:
        return x_b1, unpooling_masks

This is the unpooling function

def make_unpooling_mask(x, before_pooling):
    t = UpSampling2D()(x)

    mask = Lambda(lambda x: K.cast(K.greater(x[0],x[1]), dtype='float32'))([t, before_pooling])

    return mask 

I am getting this error

Exception has occurred: ValueError Output tensors to a Model must be the output of a Keras Layer (thus holding past layer metadata). Found: [<tf.Tensor 'lambda_1/Cast:0' shape=(?, 256, 256, 64) dtype=float32>, <tf.Tensor 'lambda_2/Cast:0' shape=(?, 128, 128, 128) dtype=float32>, <tf.Tensor 'lambda_3/Cast:0' shape=(?, 64, 64, 256) dtype=float32>, <tf.Tensor 'lambda_4/Cast:0' shape=(?, 32, 32, 512) dtype=float32>]

This happens when at the line that compiles the model model = Model(inputs, [layer, unpooling_masks], name='vgg19')

What can be done?

Answer 1

When invoking the Model API, the value for outputs argument should be tensor(or list of tensors), in this case it is a list of list of tensors, hence there is a problem. Just unpack the unpooling_masks list(*unpooling_masks) when calling Model.

model = Model(inputs, [layer, *unpooling_masks], name='vgg19')