CODEWITHSUNDEEP
kerasdeep-learningconv-neural-networkrecurrent-neural-network
prithvi poddar
prithvi poddar

Reputation: 1

How to add a ConvLSTM2D layer after a Conv2D layer?

I'm making an autoEncoder for depth estimation from monocular images. The first layer is a convolutional layer and the second layer is a convolutional LSTM layer. How do I add the ConvLSTM2D layer after the Conv2D layer.

This is the code I've tried but it gives an error.

autoencoder = Sequential()

autoencoder.add(Conv2D(64, (3, 3),strides = 2 , input_shape = (640, 480, 3), activation = 'linear'))
autoencoder.add(LeakyReLU(alpha = 0.1))

autoencoder.add(ConvLSTM2D(256, (3,3), strides = 2, input_shape = (None, 32), return_sequences = True))

I get the following error

ValueError: Input 0 is incompatible with layer conv_gr_u2d_1: expected ndim=5, found ndim=4

Upvotes: 0

Views: 829

Answers (2)

sebrockm
sebrockm

Reputation: 6012

You have maybe misunderstood what ConvLSTM2D is good for. It is designed for the scenario that you have a series of data where each data point is a picture. So, a movie would be a typical use case.

So, whatever you feed into it must have the shape (batch_size, timesteps, rows, cols, channels). On the other hand, Conv2D has an output shape of (batch_size, rows, cols, features). This is what the error is telling you.

Technically, you could just add a Reshape layer between those and generate whatever shape you want, but I don't see how this would make any sense in your scenario.

Having it vice versa (ConvLSTM2D first, then Conv2D) would make much more sense. But then you need "movie-like" input data. If I understand you correctly, you don't have that.

Upvotes: 1

Cas
Cas

Reputation: 634

input shape to Conv2D should be: 

input_shape = (batch_size, img_wd, img_hg, channels)

eg:

input_shape = (None, 640, 480, 3)

and u dont have to add input_shape argument in ConvGRU2D

Upvotes: 0

Related Questions