CODEWITHSUNDEEP
pythontensorflow
a.powell
a.powell

Reputation: 1722

Tensorflow Input Shapes Incompatible

Trying to build a Tensorflow model where my data has 70 features. Here is the setup of my first layer:

tf.keras.layers.Dense(units=50, activation='relu', input_shape=(None,70)),

Setting the input shape to (None,70) seemed best to me as I am using a feed forward neural network where each "row" of data is unique. I am using a batch size (for now) of size 10. Should my input shape change to (10,70)??

I have tried with the original (None, 70) and gotten the error:

WARNING:tensorflow:Model was constructed with shape (None, None, 70) for input Tensor("dense_33_input:0", shape=(None, None, 70), dtype=float32), but it was called on an input with incompatible shape (10, 70).

TypeError: Input 'y' of 'Mul' Op has type float64 that does not match type float32 of argument 'x'.

Quite confused on exactly what is going wrong with the input_shape as (None, 70) seems to fit the best. Any help is much appreciated.

Edit: Wanted to add a reproducible example for more context. Sorry for the length. This is a reproduction of [this example][1] to better suit my current data (non-image).

Variational Autoencoder Model

class VAE(tf.keras.Model):
    
    def __init__(self, latent_dim):
        super(VAE, self).__init__()
        self.latent_dim = latent_dim
        self.encoder = tf.keras.Sequential(
        [
            tf.keras.layers.Dense(units=50, activation='relu', input_shape=(70,)),
            tf.keras.layers.Dense(latent_dim + latent_dim), #No activation
        ])
        
        self.decoder = tf.keras.Sequential(
        [
            tf.keras.layers.Dense(units=50, activation='relu', input_shape=(latent_dim,)),
            tf.keras.layers.Dense(units=70),
        ])
        
    @tf.function
    def sample(self, eps=None):
        if eps is None:
            eps = tf.random.normal(shape=(100, self.latent_dim))
        return self.decode(eps, apply_sigmoid=True)

    def encode(self, x):
        mean, logvar = tf.split(self.encoder(x), num_or_size_splits=2, axis=1)
        return mean, logvar

    def reparameterize(self, mean, logvar):
        eps = tf.random.normal(shape=mean.shape)
        return eps * tf.exp(logvar * .5) + mean

    def decode(self, z, apply_sigmoid=False):
        logits = self.decoder(z)
        if apply_sigmoid:
            probs = tf.sigmoid(logits)
            return probs
        return logits


  [1]: https://www.tensorflow.org/tutorials/generative/cvae

Optimizer & Loss Funx

optimizer = tf.keras.optimizers.Adam(1e-4)

def log_normal_pdf(sample, mean, logvar, raxis=1):
    log2pi = tf.math.log(2. * np.pi)
    return tf.reduce_sum(
        -.5 * ((sample - mean) ** 2. * tf.exp(-logvar) + logvar + log2pi), axis=raxis)

def compute_loss(model, x):
    mean, logvar = model.encode(x)
    z = model.reparameterize(mean, logvar)
    x_logit = model.decode(z)
    cross_ent = tf.nn.sigmoid_cross_entropy_with_logits(logits=x_logit, labels=x)
    logpx_z = -tf.reduce_sum(cross_ent, axis=[1])
    logpz = log_normal_pdf(z, 0, 0)
    logqz_x = log_normal_pdf(z, mean, logvar)
    return -tf.reduce_mean(logpx_z + logpz - logqz_x)

@tf.function
def train_step(model, x, optimizer):
    with tf.GradientTape() as tape:
        loss = compute_loss(model, x)
    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))

Train

X = tf.random.uniform((100,70))
y = tf.random.uniform((100,))

ds_train = tf.data.Dataset.from_tensor_slices((X, y))

tf.random.set_seed(1)

train = ds_train.shuffle(buffer_size=len(X))
train = train.batch(batch_size=10, drop_remainder=False)

epochs = 5
latent_dim = 2

model = VAE(2)

for epoch in range(1, epochs+1):
    start_time = time.time()
    for i, (train_x, train_y) in enumerate(train):
        train_step(model, train_x, optimizer)
    end_time = time.time()
    
    loss = tf.keras.metrics.Mean()
    for i, (test_x, test_y) in enumerate(ds_test):
        loss(compute_loss(model, test_x))
    elbo = -loss.result()
    display.clear_output(wait=False)
    print('Epoch: {}, Test set ELBO: {}, time elapse for current epoch: {}'
         .format(epoch, elbo, end_time - start_time))

Upvotes: 1

Views: 2481

Answers (1)

jkr
jkr

Reputation: 19260

The input_shape should not include the batch dimension. Use input_shape=(70,).

tf.keras.layers.Dense(units=50, activation='relu', input_shape=(70,))

You can set the batch size when you call model.fit(..., batch_size=10). See the documentation on tf.keras.Model.fit.

There was another error in the original post, due to passing an int32 value to tf.math.exp. That line should should read

logpz = log_normal_pdf(z, 0., 0.)

to solve that error. Note the 0. values, which evaluate to floats instead of ints.

Upvotes: 1

Related Questions