CODEWITHSUNDEEP
python-2.7tensorflow
Glrs
Glrs

Reputation: 1080

tf.initialize_variables() inconvenience - FailedPreconditionError tensorflow

I want to be able to initialize tensorflow variables with a specific list using tf.initialize_variables(). So, to test the case I am trying this but it fails:

sess.run(tf.initialize_variables(tf.all_variables()))

with the error:

tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value Layer_0/Initialize_Variables/weights

What is really weird to me, is that it works this way:

sess.run(tf.initialize_variables([tf.all_variables()[0]]))
sess.run(tf.initialize_variables([tf.all_variables()[1]]))
sess.run(tf.initialize_variables([tf.all_variables()[2]]))
...

So, the same happens if I create my own list of variables that I want to initialize. Probably I am misunderstanding something of the tensorflow mechanisms. Any ideas? I'd be glad if I don't have to do that using a loop for each variable.

Upvotes: 0

Views: 941

Answers (1)

mrry
mrry

Reputation: 126184

It's hard to tell without seeing the code, but my guess is that you're initializing one variable from another variable, which is a little tricky in TensorFlow. For example, I imagine you're doing something like this:

v_1 = tf.Variable(...)
v_2 = tf.Variable(v_1)

This will fail if you try to do sess.run(tf.initalize_all_variables()) because it will attempt to initialize v_1 and v_2 in parallel, but the initialization of v_2 will fail (because its input is currently uninitialized).

If you iterate over the tf.all_variables() list in increasing order, it will work, because v_1 is added to the list before v_2, so running things sequentially will cause v_1 to be initialized before v_2. (At a guess, if you iterated over tf.all_variables() in reverse order, it would fail with a similar error.)

The fix is a bit subtle: any time you initialize a variable v_2 from another variable v_1, you should use v_1.initialized_value() as the argument to v_2's constructor:

v_1 = tf.Variable(...)
v_2 = tf.Variable(v_1.initialized_value())

Using v_1.initialized_value adds a control dependency between the two initializers, so that v_1 is initialized before v_2, even if you try to initialize them in parallel.

Upvotes: 2

Related Questions