CODEWITHSUNDEEP
pythonnumpytensorflowkeras
asaf92
asaf92

Reputation: 1855

Why do you have to reshape inputs in Keras/Tensorflow 2?

I have a simple network:

input_layer = Input(1)
inner_layer = Dense(4, activation='relu')(input_layer)
output_layer = Dense(1, activation='linear')(inner_layer)
model = Model(input_layer, output_layer)
optimizer = Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')

Intuitively, inference for input 0 would simply be model.predict(0). However this generates this error: expected input_2 to have 2 dimensions, but got array with shape ()

I understand it expects the input (which is a single number) to be two dimensional, but I don't understand what Tensorflow accepts as valid input. I tried many different combinations of inputs, some work and some don't, it seems quite inconsistent and the warnings/error are usually not useful:

When calling model.predict():

When calling model() (I saw here that's needed to get the gradients):

When using np.reshape:

What seems to be working consistently is using numpy's reshape function. it always works both for model.predict() and for model() on all the inputs as long as they're reshaped to a [1,1] shape.

My questions:

  1. What are the guidelines to feeding inputs into tensorflow models in regards to inputs shapes/types?
  2. What does "shape ()" mean?
  3. What does "(None, 1)" mean?
  4. Why does reshape work but [[0]] does not? Both create a 2-dimensional collection.
  5. Why when calling model(0)/model([0])/model([[0]]) does this warning show: WARNING:tensorflow:Model was constructed with shape Tensor("input_1:0", shape=(None, 1), dtype=float32) for input (None, 1), but it was re-called on a Tensor with incompatible shape ()?

Upvotes: 2

Views: 3161

Answers (1)

Vlad
Vlad

Reputation: 8585

The shape of the tensor inputs = tf.keras.layers.Input(1) is (None, 1) (run inputs.get_shape().as_list()). The None means any size that is determined dynamically (batch size). The 1 is the shape of your data point. For example, this is a tensor of shape (3, 1):

[[1], [2], [1]]

This is a tensor of shape (3,)

[1, 2, 1]

If you define a tensor of shape (None, 1) you must feed the data of same shape.

The [[0]] has correct shape (1, 1) and won't throw any error or warning if you pass it as numpy array of expected data type:

import tensorflow as tf
import numpy as np

input_layer = tf.keras.layers.Input(1)
inner_layer = tf.keras.layers.Dense(4, activation='relu')(input_layer)
output_layer = tf.keras.layers.Dense(1, activation='linear')(inner_layer)
model = tf.keras.models.Model(input_layer, output_layer)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')
print(model(np.array([[0.]], dtype=np.float32)).numpy()) # [[0.]]
print(model.predict(np.array([[0.], [1]], dtype=np.float32))) # [[0.        ]
                                                              # [0.08964952]]

np.reshape() works because it automatically converts your list to numpy array. For more about np.reshape refer to the official documentation.

The model.predict() also expects the same shape as model.__call__(), but can perform automatic reshaping (expands dimension on the left, i.e [1] -- > [[1]]).

Upvotes: 4

Related Questions