CODEWITHSUNDEEP
pythontensorflow
Daniel Renshaw
Daniel Renshaw

Reputation: 34187

TensorFlow supervisor prevents variable assignment: Graph is finalized and cannot be modified

This code works fine:

import tensorflow as tf
x = tf.Variable(initial_value=0)

with tf.Session() as session:
    print session.run(x.assign(1))

But this code fails:

import tensorflow as tf
x = tf.Variable(initial_value=0)
supervisor = tf.train.Supervisor(logdir="/tmp")

with tf.Session() as session:
    print session.run(x.assign(1))

The only difference is the instantiation of a tf.train.Supervisor. Note that we don't even use the supervisor to create a managed session.

The error is:

python tf_supervisor_freeze.py
Traceback (most recent call last):
  File "tf_supervisor_freeze.py", line 6, in <module>
    print session.run(x.assign(1))
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 522, in assign
    return state_ops.assign(self._variable, value, use_locking=use_locking)
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign
    use_locking=use_locking, name=name)
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 491, in apply_op
    preferred_dtype=default_dtype)
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 702, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 110, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 103, in constant
    attrs={"value": tensor_value, "dtype": dtype_value}, name=name).outputs[0]
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2286, in create_op
    self._check_not_finalized()
  File "<virtual_env_path>/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2009, in _check_not_finalized
    raise RuntimeError("Graph is finalized and cannot be modified.")
RuntimeError: Graph is finalized and cannot be modified.

Process finished with exit code 1

Error remains if tf.train.Supervisor(logdir="/tmp", summary_op=None, saver=None) is used to disable some of the supervisor's services.

This problem was raised by somebody else on Github but no answer was provided there; the request was to raise the issue on StackOverflow instead. The only relevant StackOverflow question does not appear to address this specific case.

Upvotes: 4

Views: 3951

Answers (3)

Kenneth
Kenneth

Reputation: 91

An assign operation is a new operation in the Graph. If that operation is defined before Graph finalization (even if executed later), then it's OK.

Executing the operation effectively assigns a value to the variable. This can safely be done after Graph finalization, as it doesn't change the graph.

Bottomline is: - declare the ops before graph finalization - run them after.

Upvotes: 2

LF-DevJourney
LF-DevJourney

Reputation: 28534

Just as the error says, you cannot modify the graph when the graph is finalized.

RuntimeError("Graph is finalized and cannot be modified.")

When execute this code tf.train.Supervisor(), the graph will be finalized. You can check code of it with path /tensorflow/python/training/supervisor.py in tensorflow of version 1.0.

And you will find this:

# The graph is not allowed to change anymore.
graph.finalize()

So you cannot modify the graph after Supervisor().

Upvotes: 4

Daniel Renshaw
Daniel Renshaw

Reputation: 34187

I can't explain why but I've found a work-around. This works:

import tensorflow as tf
x = tf.Variable(initial_value=0)
a = x.assign(1)
supervisor = tf.train.Supervisor(logdir="/tmp")

with tf.Session() as session:
    print session.run(a)

All that has been done is to move the creation of the assignment operation to before the creation of the supervisor (i.e. a = x.assign(1)). It appears the supervisor really does freeze the graph preventing one from creating, or more accurately using, new operations after the supervisor's creation.

Upvotes: 3

Related Questions