CODEWITHSUNDEEP
pythontensorflow
zxcv
zxcv

Reputation: 153

Why does my tensorflow code runs more and more slower as the training goes on?

import tensorflow as tf

input_data = tf.constant([[1.,1.]])
output_data = tf.constant([[1.,0.]])
weight = tf.Variable([[1.,1.],
                      [1.,1.]])

sess = tf.Session()
sess.run(tf.global_variables_initializer())
optimizer = tf.train.GradientDescentOptimizer(0.1)

for epoch in range(1000):
    y = tf.matmul(input_data, weight)
    loss = (output_data[0][0] - y[0][0])**2 + (output_data[0][1] - y[0][1])**2
    sess.run(optimizer.minimize(loss))
    print(epoch)

The above program gets more and more slower as the epoch goes on. I think it's because new nodes are keep being added for each epoch. How do I handle this?

Upvotes: 1

Views: 100

Answers (1)

bivouac0
bivouac0

Reputation: 2560

Try this...

import time
import tensorflow as tf

input_data = tf.constant([[1.,1.]])
output_data = tf.constant([[1.,0.]])
weight = tf.Variable([[1.,1.],
                      [1.,1.]])

optimizer = tf.train.GradientDescentOptimizer(0.1)
y = tf.matmul(input_data, weight)
loss = (output_data[0][0] - y[0][0])**2 + (output_data[0][1] - y[0][1])**2
train = optimizer.minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print('Initial weights: ', sess.run(weight))
    for epoch in range(1000):
        st = time.time()
        sess.run(train)
        print('Epoch %3d : %.3f ms' %(epoch, 1e3*(time.time()-st)))
    print('Weights: ', sess.run(weight))

The original code is recreating the graph each epoch. If you do it this way, the graph is only created once and the only work in the loop is the gradient calculations/updates.

Upvotes: 3

Related Questions