CODEWITHSUNDEEP
pythontensorflowmachine-learningdependenciescontrol-flow
K.Wanter
K.Wanter

Reputation: 1059

Very strange tensorflow behavior

I have very simple lines that produce very strange unexpected behavior:

import tensorflow as tf

y = tf.Variable(2, dtype=tf.int32)

a1 = tf.assign(y, y + 1)
a2 = tf.assign(y, y * 2)

with tf.control_dependencies([a1, a2]):
    t = y+0

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for i in range(4):
        print('t=%d' % sess.run(t))
        print('y=%d' % sess.run(y))

What is expected is 

t=6
y=6
t=14
y=14
t=30
y=30
t=62
y=62

But first run, I got:

t=6
y=6
t=13
y=13
t=26
y=26
t=27
y=27

Second run, I got:

t=3
y=3
t=6
y=6
t=14
y=14
t=15
y=15

Third run, I got:

t=6
y=6
t=14
y=14
t=28
y=28
t=56
y=56

Very ridiculous, multiple runs produces multiple different output sequences, very strange, can someone help?

EDIT: changing to 

import tensorflow as tf
import os
y = tf.Variable(2, dtype=tf.int32)

a1 = tf.assign(y, y + 1)
a2 = tf.assign(y, y * 2)
a3 = tf.group(a1, a2)
with tf.control_dependencies([a3]):
    t = tf.identity(y+0)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for i in range(4):

        print('t=%d' % sess.run(t))
        print('y=%d' % sess.run(y))

...still doesn't work properly.

It is still strange that this code:

a1 = tf.assign(y, y + 1)
with tf.control_dependencies([a1]):
  a2 = tf.assign(y, y * 2)
  with tf.control_dependencies([a2]):
    t = tf.identity(y)

... works properly, but simply move a2 to before as

a1 = tf.assign(y, y + 1)
a2 = tf.assign(y, y * 2)
with tf.control_dependencies([a1]):
  with tf.control_dependencies([a2]):
    t = tf.identity(y)

... it doesn't.

Upvotes: 2

Views: 84

Answers (1)

Maxim
Maxim

Reputation: 53758

The problem with your approach is that the order of a1 and a2 matters too: you want a1 to be evaluated before a2. tf.control_dependencies([a1, a2]) guarantees that t is performed after both a1 and a2, but they themselves can be evaluated in any order.

I'd go with explicit dependency like that:

y = tf.Variable(2, dtype=tf.int32)
a1 = tf.assign(y, y + 1)
with tf.control_dependencies([a1]):
  a2 = tf.assign(y, y * 2)
  with tf.control_dependencies([a2]):
    t = tf.identity(y)

with tf.Session() as sess:
  sess.run(tf.global_variables_initializer())
  for i in range(4):
    print('t=%d' % sess.run(t))
    print('y=%d' % sess.run(y))

Output:

t=6
y=6
t=14
y=14
t=30
y=30
t=62
y=62

Upvotes: 1

Related Questions