CODEWITHSUNDEEP
pythonnumpytensorflow
ESala
ESala

Reputation: 7058

TensorFlow: Linear Regression with multiple inputs returns NaNs

This is my first attemp at TensorFlow: I am building a Linear Regression model with multiple inputs.

The problem is that the result is always NaN, and I suspect that it is because I am a complete noob with matrix operations using numpy and tensorflow (matlab background hehe).

Here is the code:

import numpy as np
import tensorflow as tf

N_INP = 2
N_OUT = 1

# Model params
w = tf.Variable(tf.zeros([1, N_INP]), name='w')
b = tf.Variable(tf.zeros([1, N_INP]), name='b')

# Model input and output
x = tf.placeholder(tf.float32, [None, N_INP], name='x')
y = tf.placeholder(tf.float32, [None, N_OUT], name='y')
linear_model = tf.reduce_sum(x * w + b, axis=1, name='out')

# Loss as sum(error^2)
loss = tf.reduce_sum(tf.square(linear_model - y), name='loss')

# Create optimizer
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss, name='train')

# Define training data
w_real = np.array([-1, 4])
b_real = np.array([1, -5])
x_train = np.array([[1, 2, 3, 4], [0, 0.5, 1, 1.5]]).T
y_train = np.sum(x_train * w_real + b_real, 1)[np.newaxis].T
print('Real X:\n', x_train)
print('Real Y:\n', y_train)

# Create session and init parameters
sess = tf.Session()
sess.run(tf.global_variables_initializer())

# Training loop
train_data = {x: x_train, y: y_train}
for i in range(1000):
    sess.run(train, train_data)

# Eval solution
w_est, b_est, curr_loss, y_pred = sess.run([w, b, loss, linear_model], train_data)
print("w: %s b: %s loss: %s" % (w_est, b_est, curr_loss))
print("y_pred: %s" % (y_pred,))

And here is the output:

Real X:
 [[ 1.   0. ]
 [ 2.   0.5]
 [ 3.   1. ]
 [ 4.   1.5]]
Real Y:
 [[-5.]
 [-4.]
 [-3.]
 [-2.]]

w: [[ nan  nan]] b: [[ nan  nan]] loss: nan
y_pred: [ nan  nan  nan  nan]

Upvotes: 2

Views: 852

Answers (1)

Miriam Farber
Miriam Farber

Reputation: 19634

You need to add keep_dims=True inside your definition of linear_model. That is,

linear_model = tf.reduce_sum(x * w + b, axis=1, name='out',keep_dims=True)

The reason is that otherwise the result is "flattened", and you cannot subtract y from it.

For example,

'x' is [[1,2,3],
        [4,5,6]]   
tf.reduce_sum(x, axis=1) is [6, 15]   
tf.reduce_sum(x, axis=1, keep_dims=True) is [[6], [15]]

Upvotes: 1

Related Questions