CODEWITHSUNDEEP
pythontensorflowkeras
WX_M
WX_M

Reputation: 488

Keras Unet: A `Concatenate` layer requires inputs with matching shapes except for the concat axis Error off by 1

I have skimmed through the many threads that have this same sort of error, but none seem to be similar to the issue I am facing. Below is a simple Unet architecture:

tdata = np.zeros([100,500,500,5])

def unet2(input_data):

    inputs = layers.Input((input_data.shape[1],input_data.shape[2],input_data.shape[3]))

    conv1 = layers.Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
    conv1 = layers.Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)
    pool1 = layers.MaxPooling2D(pool_size=(2, 2))(conv1)
    conv2 = layers.Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)
    conv2 = layers.Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)
    pool2 = layers.MaxPooling2D(pool_size=(2, 2))(conv2)
    conv3 = layers.Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)
    conv3 = layers.Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)
    pool3 = layers.MaxPooling2D(pool_size=(2, 2))(conv3)
    conv4 = layers.Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)
    conv4 = layers.Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)
    drop4 = layers.Dropout(0.5)(conv4)
    pool4 = layers.MaxPooling2D(pool_size=(2, 2))(drop4)

    conv5 = layers.Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)
    conv5 = layers.Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)
    drop5 = layers.Dropout(0.5)(conv5)

    up6 = layers.Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(layers.UpSampling2D(size = (2,2))(drop5))
    merge6 = layers.concatenate([drop4,up6], axis = 3)
    conv6 = layers.Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)
    conv6 = layers.Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)

    up7 = layers.Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(layers.UpSampling2D(size = (2,2))(conv6))
    merge7 = layers.concatenate([conv3,up7], axis = 3)
    conv7 = layers.Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)
    conv7 = layers.Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)

    up8 = layers.Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(layers.UpSampling2D(size = (2,2))(conv7))
    merge8 = layers.concatenate([conv2,up8], axis = 3)
    conv8 = layers.Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)
    conv8 = layers.Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)

    up9 = layers.Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(layers.UpSampling2D(size = (2,2))(conv8))
    merge9 = layers.concatenate([conv1,up9], axis = 3)
    conv9 = layers.Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)
    conv9 = layers.Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
    conv9 = layers.Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
    conv10 = layers.Conv2D(1, 1, activation = 'sigmoid')(conv9)

    model = Model(input = input_data, output = conv10)

    model.compile(optimizer = Adam(lr = 1e-4), loss = 'binary_crossentropy', metrics = ['accuracy'])

I then simply run the unet as:

unet2(tdata)

where tdata represents a sample dataset that I might have that would be ready for the Unet. Here, the dimensions are [N,x,y,v], where N = number of samples, x = x-dimension, y = y-dimension, and v = number of variables. In other words, I have 100 samples of 500x500 data consisting of 5 variables. When I run this exact code I get the error:

ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 125, 125, 256), (None, 124, 124, 256)]

at the 'merge7' part. Therefore, there is an offset of 1 issue somewhere that I cannot seem to pinpoint - why is this issue occurring and how best to fix this?

My versions of Keras are 2.4.0 and tf = 2.3.1

Upvotes: 3

Views: 2200

Answers (1)

xdurch0
xdurch0

Reputation: 10475

Think about the sizes of the feature maps. 500x500 downsamples to 250x250, okay. That downsamples to 125x125, okay. But what do you get when you downsample 125x125 by factor 2 (as the max pooling layers do)? Can you have feature maps with fractions of sizes (no)? Accordingly, this will result in feature maps of size 62x62 (basically cutting off one pixel in the input), which will later upsample back to 124x124 and create a mismatch.

You can fix this by carefully controlling the padding in the hidden layers, but by far the easiest option would be to either pad or resize your inputs to a power of 2, or at least a number that is divisible by the overall downscaling factor of the network (16 if I'm not mistaken). In your case, the closest power of 2 is 512, which is very close to the input size, so it should be okay for this use case.

Concretely, all you should have to change in this toy example is tdata = np.zeros([100,512,512,5]).

Upvotes: 3

Related Questions