CODEWITHSUNDEEP
pythonkerasautoencoder
Travasaurus
Travasaurus

Reputation: 654

ValueError: Error when checking input: expected input_58 to have 3 dimensions, but got array with shape (10000, 10020)

So I am trying to build a 1D CNN autoencoder with data that has the shape (10000, 10020). Here is my setup:

input_wave = Input(shape=(10020,1))
encode1 = Conv1D(16, 16)(input_wave)
pool1 = MaxPooling1D(3)(encode1)
encode2 = Conv1D(32, 60)(pool1)
pool2 = MaxPooling1D(5)(encode2)
encode3 = Conv1D(64, 130)(pool2)
pool3 = MaxPooling1D(5)(encode3)
encode4 = Conv1D(128, 105)(pool3)
up1 = UpSampling1D(10)(encode4)
drop = Dropout(.2)(up1)
up2 = UpSampling1D(10)(drop)
drop2 = Dropout(.2)(up2)
flat  = Flatten()(drop2)
Den = Dense(10020, activation = 'sigmoid', input_shape = (12800, 1))(flat)

autoencoder.compile(optimizer = 'Adam',
                    loss = 'mean_squared_error',
                    metrics = ['accuracy'])

autoencoder = Model(input_wave, Den)
autoencoder.summary()

Here is the summary

I unfortunately cannot post the data itself, but its numeric, between 0 and 1, has no NAs, and is waveform data.

When I run data.shape(), I get:

(10000, 10020)

When I run type(data) I get:

pandas.core.frame.DataFrame

I my fit statement looks like this:

autoencoder.fit(data,data,
                batch_size = 250,
                epochs = 10)

and I get the following error:

  ValueError: Error when checking target: expected dense_7 to have 2 dimensions, but got array with shape (10000, 10020, 1)

I have attempted to use data.values, and reshaping my data with the following code:

data = np.reshape(i_data[1].values, (10000, 10020, 1))

but that gives me this error:

ValueError: Error when checking target: expected dense_7 to have 2 dimensions, but got array with shape (10000, 10020, 1)

So it seems that my network is structured poorly either way. I'm sorry, but I cannot release my data :( but I can make toy data if that will help! Anyone have any ideas where my structure issue is?

Upvotes: 0

Views: 72

Answers (1)

Thibault Bacqueyrisses
Thibault Bacqueyrisses

Reputation: 2331

First of all, your autoencoder is strangely defined, i don't know what you are using for exemple.

You have to reconstruct your data with the same shape, so your last layer need to output a shape of (batch, 10020, 1), this is you error, you are applying Flatten and Dense to your Data, that's why you have only 2 Dimension as expected output.

Here is a simple working exemple of what you can have as Autoencoder : 

x = Input(shape=(10020, 1), name="input")
h = Conv1D(filters=50, kernel_size=3, activation="relu", padding='same', name='Conv1')(x)
h = MaxPooling1D(pool_size=2, name='Maxpool1')(h)
h = Conv1D(filters=150, kernel_size=3, activation="relu", padding='same', name='Conv2')(h)
h = MaxPooling1D(pool_size=2,  name="Maxpool2")(h)
y = Conv1D(filters=150, kernel_size=3, activation="relu", padding='same', name='conv-decode1')(h)
y = UpSampling1D(size=2, name='upsampling1')(y)
y = Conv1D(filters=50, kernel_size=3, activation="relu", padding='same', name='conv-decode2')(y)
y = UpSampling1D(size=2, name='upsampling2')(y)
y = Conv1D(filters=1, kernel_size=3, activation="relu", padding='same', name='conv-decode3')(y)


autoencoder = Model(x, y)

You can play with the kernels size and the filters as you wish, but you need to decode your data the same way you encoded it.

Moreover here your model have 129 millions parameters because of your last Dense layer, this is wayyyy to much.

Tell me if you need more precisions.

Upvotes: 1

Related Questions