CODEWITHSUNDEEP
pythontensorflowneural-network
solver149
solver149

Reputation: 425

Tensorflow multiple session.run( ) in same iteration

Following are 2 code patterns for the training part of Tensorflow NN.

I find it logical to use the model 1. But I am seeing model 2 in multiple places frequently. I feel Model 2 is wrong. Wont model run the graph in session twice for every iteration for the same data? Is there something I am missing and people do it for any other reasons?

Model 1

for epoch in range(epochs):
    for iteration in range(num_tr_iter):
        _, loss, accuracy = sess.run([optimizer, loss, accuracy], feed_dict)

Model 2

for epoch in range(epochs):
    for iteration in range(num_tr_iter):
        sess.run(optimizer, feed_dict)
        loss, accuracy = sess.run([loss, accuracy],feed_dict)

EDIT : Am expanding the question for more clarity

If the below sess.run() execute optimizer node, it will execute all its dependent node. It will run the underlying convnet and loss function as well.

    sess.run(optimizer, feed_dict)

Next if the below sess.run() execute the loss node, why doesnt it execute the convnet with its current weights. I am not inferring that it will run the optimization again. Even to arrive at the current loss, wouldnt tensorflow execute the convnet and calculate the loss?

        loss, accuracy = sess.run([loss, accuracy],feed_dict)

Upvotes: 3

Views: 2138

Answers (2)

Jacob Soderlund
Jacob Soderlund

Reputation: 329

Session.Run will only run calculate the graph element(s) given to in the first argument.

So in Model 2, sess.run(optimizer, feed_dict) just applies the weight updates for the model, and loss, accuracy = sess.run([loss, accuracy],feed_dict) just calculates the loss and accuracy on the model after weights have been updated. The whole computation isn't being run twice since these operations are independent of each other.

Note that the Session.Run docs mention that:

Order in which... operations are evaluated inside the call is undefined.

So Model 1 might not even give an accurate loss and accuracy, since it's unclear if it was measured before or after the weight update. (In practice this might be fine, though)

Upvotes: 2

Abhishek Verma
Abhishek Verma

Reputation: 1

In model 2, the first line of code runs the optimizer on the model, and the second line calculates the loss and accuracy. It is not running the graph twice.

Model 1 and Model 2 are essentially the same, but points go to model 1 for brevity.

Upvotes: 0

Related Questions