CODEWITHSUNDEEP
pythontensorflow
user1357015
user1357015

Reputation: 11686

Interpretation regarding session in tensorflow

Consider the following tensorflow code from the example:

import numpy as np
import tensorflow as tf

# Model parameters
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W * x + b
y = tf.placeholder(tf.float32)
# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizer
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
# training data
x_train = [1,2,3,4]
y_train = [0,-1,-2,-3]
# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
  sess.run(train, {x:x_train, y:y_train})

# evaluate training accuracy
curr_W, curr_b, curr_loss  = sess.run([W, b, loss], {x:x_train, y:y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

I am ok until the for loop at the end.

I'm confused about what exactly happens in the sess.run(train, {x:x_train, y:y_train} part. As far as I can tell it keeps passing x = [1,2,3,4] and y = [0,-1,-2,-3] to be trained on.

Does each loop then update W and b? It then keeps passing the same two x and y and every time iterates from the last version of W and b?

Two questions:

1) Where does the update happen or is that implicit in sess.run? Does sess.run(train... know specifically to updated W and b because those are variables?

2) Is it ok to keep passing the same x and y in every time? -- This question only makes sense if I'm incorrect in my thoughts regarding question 1. If it updates W,b every time automatically, then it completely makes sense, we just keep going until we find the best fit. However if the update doesn't happen automatically then I'm not sure...

Upvotes: 3

Views: 114

Answers (2)

ilyakhov
ilyakhov

Reputation: 1319

Does each loop then update W and b?

Yes.

It then keeps passing the same two x and y and every time iterates from the last version of W and b?

Yes.

1) Where does the update happen or is that implicit in sess.run?

The main concept of TensorFlow pipeline is the computation graph. All code above is just initialization of it. Yes, it looks misleading: your wrote Python code, but it doesn't work like Python. But it's okay, because it's just Python API. All magic happenes in C++ core of Tensorflow. And the heart of this magic is the computation graph. Try to learn more in the official manuals or ask me in the comments.

Or shorter and simply: yes, all updates are made implicitly.

Does sess.run(train... know specifically to updated W and b because those are variables?

Yes.

Upvotes: 1

Nabeel Ahmed
Nabeel Ahmed

Reputation: 19252

A little background:

Above the you just initialized x and y using tf.placeholder(), and b and W as tf.Variable().

As the name say placeholder is a promise to provide a value later i.e.

Variable are simply the training parameters (W(matrix), b(bias) same as the normal variables we use in daily programming, which the trainer updates/modify on each run/step.

While placeholder doesn't require any initial value, that when you created x and y TF doesn't allocated any memory, instead later when you feed the placeholders in the sess.run() using feed_dict, TensorFlow will allocate the appropriately sized memory for them (x and y) - this unconstrained-ness allows us to feed any size and shape of data.

OP's questions:

Where does the update happen or is that implicit in sess.run? Does sess.run(train... know specifically to updated W and b because those are variables?

Yes - the first argument to run() is the trainer i.e. GradientDescentOPtimizer, and second is the feed_dict argument:

session.run(trainer, {feed_dict})

Typical feed_dict arg 'd be - each value from feed will provided concrete value to the placeholder on each step/run, it takes the form:

{placeholder: feed}
or,
{<placeholder>: <tensor of values to be passed for placeholder>}

Is it ok to keep passing the same x and y in every time?

For every step x and y will be different i.e. an item from the feed (training data set). For example:

a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b  # + provides a shortcut for tf.add(a, b)

sess.run():

print(sess.run(adder_node, {a: 3, b:4.5}))
print(sess.run(adder_node, {a: [1,3], b: [2, 4]}))

resulting in the output

7.5
[ 3.  7.]

In the first case 3 and 4.5 will be passed to a and b respectively, and then to adder_node ouputting 7. In second case there's a feed list, first step 1 and 2 will be added, next 3 and 4 (a and b).

Upvotes: 1

Related Questions