ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis

Question

I'm trying to implement a special type of neural network with Keras functional API, as seen below:

But I'm having a problem with the concatenate layer:

ValueError: A "Concatenate" layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 160, 160, 384), (None, 160, 160, 48)]

Notice: From my research I assume that this question is not duplicate, I've seen this question, and this post (translated with Google), but they don't seem to work (instead, they make problems even slightly "worse").

---

Here's the code of the neural network before concat layer:

from keras.layers import Input, Dense, Conv2D, ZeroPadding2D, MaxPooling2D, BatchNormalization, concatenate
from keras.activations import relu
from keras.initializers import RandomUniform, Constant, TruncatedNormal

#  Network 1, Layer 1
screenshot = Input(shape=(1280, 1280, 0), dtype='float32', name='screenshot')
# padded1 = ZeroPadding2D(padding=5, data_format=None)(screenshot)
conv1 = Conv2D(filters=96, kernel_size=11, strides=(4, 4), activation=relu, padding='same')(screenshot)
# conv1 = Conv2D(filters=96, kernel_size=11, strides=(4, 4), activation=relu, padding='same')(padded1)
pooling1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='same')(conv1)
normalized1 = BatchNormalization()(pooling1)  # https://stats.stackexchange.com/questions/145768/importance-of-local-response-normalization-in-cnn

# Network 1, Layer 2

# padded2 = ZeroPadding2D(padding=2, data_format=None)(normalized1)
conv2 = Conv2D(filters=256, kernel_size=5, activation=relu, padding='same')(normalized1)
# conv2 = Conv2D(filters=256, kernel_size=5, activation=relu, padding='same')(padded2)
normalized2 = BatchNormalization()(conv2)
# padded3 = ZeroPadding2D(padding=1, data_format=None)(normalized2)
conv3 = Conv2D(filters=384, kernel_size=3, activation=relu, padding='same',
               kernel_initializer=TruncatedNormal(stddev=0.01),
               bias_initializer=Constant(value=0.1))(normalized2)
# conv3 = Conv2D(filters=384, kernel_size=3, activation=relu, padding='same',
#               kernel_initializer=RandomUniform(stddev=0.1),
#               bias_initializer=Constant(value=0.1))(padded3)

# Network 2, Layer 1

textmaps = Input(shape=(160, 160, 128), dtype='float32', name='textmaps')
txt_conv1 = Conv2D(filters=48, kernel_size=1, activation=relu, padding='same',
                   kernel_initializer=TruncatedNormal(stddev=0.01), bias_initializer=Constant(value=0.1))(textmaps)

# (Network 1 + Network 2), Layer 1

merged = concatenate([conv3, txt_conv1], axis=1)

This is how interpreter evaluates variables conv3 and txt_conv1:

>>> conv3
<tf.Tensor 'conv2d_3/Relu:0' shape=(?, 160, 160, 384) dtype=float32>
>>> txt_conv1
<tf.Tensor 'conv2d_4/Relu:0' shape=(?, 160, 160, 48) dtype=float32>

This is how the interpreter evaluates txt_conv1 and conv3 variables after setting image_data_format to channels_first:

>>> conv3
<tf.Tensor 'conv2d_3/Relu:0' shape=(?, 384, 160, 0) dtype=float32>
>>> txt_conv1
<tf.Tensor 'conv2d_4/Relu:0' shape=(?, 48, 160, 128) dtype=float32>

Both of the layers have shapes which are not actually described in the architecture.

---

Is there any way to solve this problem? Maybe I didn't write the appropriate code (I'm new to Keras).

P.S

I know that the code above is not organized, I'm just testing.

Thank you!

Answer 1

You should change the axis to -1 in the concatenate layer since the shapes of the two tensors that you want to concatenate only differ in their last dimension. The resulting tensor will then be of shape (?, 160, 160, 384 + 48).