CODEWITHSUNDEEP
pythontensorflowregression
Mersper
Mersper

Reputation: 97

How to do the regression by tensorflow in this example?

I am using tensorflow to do a linear regression. Here I am facing a problem:

import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (8,6)

data = pd.read_csv('./data.csv')
xs = data["A"][:100]
ys = data["B"][:100]

X = tf.placeholder(tf.float32, name='X')
Y = tf.placeholder(tf.float32, name='Y')

W = tf.Variable(tf.random_normal([1]),name = 'weight')
b = tf.Variable(tf.random_normal([1]),name = 'bias')

Y_pred = tf.add(tf.multiply(X,W), b)

sample_num = xs.shape[0]
loss = tf.reduce_sum(tf.pow(Y_pred - Y,2))/sample_num
learning_rate = 0.0001
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)

n_samples = xs.shape[0]
init = tf.global_variables_initializer()
with tf.Session() as sess:

    sess.run(init)

    for i in range(100):
        for x,y in zip(xs,ys):
            _, l = sess.run([optimizer, loss], feed_dict={X: x, Y:y})

    W, b = sess.run([W, b])

plt.plot(xs, ys, 'bo', label='Real data')
plt.plot(xs, xs*W + b, 'r', label='Predicted data')
plt.legend()
plt.show()

The data.csv is here.

The plot is diametrically opposed to what I expected: enter image description here So, what is the problem? I am a beginner of python and tensorflow, and just can't reach the points.

Upvotes: 1

Views: 59

Answers (2)

Nipun Wijerathne
Nipun Wijerathne

Reputation: 1829

Since your learning rate and the number of epochs are too small, your regression models haven't converged. Therefore, you may need to increase the learning rate and use the tf.train.AdamOptimizer.

Here I set the learning rate to 2, epochs=10000 and got the following graph.

enter image description here

Here I have given the code with the comments where necessary.

import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt

plt.rcParams["figure.figsize"] = (8, 6)

data = pd.read_csv('./data.csv')
xs = data["A"][:100]
ys = data["B"][:100]

X = tf.placeholder(tf.float32, name='X')
Y = tf.placeholder(tf.float32, name='Y')

W = tf.Variable(tf.random_normal([1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')

Y_pred = tf.add(tf.multiply(X, W), b)

loss = tf.reduce_mean(tf.pow(Y_pred - Y, 2))
learning_rate = 2 #increase the learning rate
optimizer = tf.train.AdamOptimizer(learning_rate).minimize(loss)#use the AdamOptimizer

BATCH_SIZE = 8 #Batch Size define here

n_samples = xs.shape[0]
init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)

    for i in range(10000): #increase the num of epoches
        for start, end in zip(range(0, n_samples, BATCH_SIZE), # mini batch Gradientdecent
                              range(BATCH_SIZE, n_samples + 1, BATCH_SIZE)):
            _, l = sess.run([optimizer, loss], feed_dict={X: xs[start:end], Y: ys[start:end]})

    prediction = sess.run(Y_pred, feed_dict={X: xs})
    #W, b = sess.run([W, b])

plt.plot(xs, ys, 'bo', label='Real data')
plt.plot(xs, prediction, 'r', label='Predicted data')
plt.legend()
plt.show()

Also, you can use the mini batch gradientdescent method to accelerate the convergence as the code above.

Moreover, you can increase the number of epochs and learning rate further to get the optimal result.

Hope this helps.

Upvotes: 1

user4340135
user4340135

Reputation:

As Nipun mentioned, try AdamOptimizer instead of GradientDescentOptimizer.

You will often find that AdamOptimizer is generally a better optimizer than GradientDescentOptimizer and reaches the minima much faster.

It does so by adapting the learning rate instead of keeping it constant (0.0001 in your case).

Also, more the number of epochs, better the model (not considering over-fitting here).

Upvotes: 1

Related Questions