CODEWITHSUNDEEP
tensorflow
Mohammed AlQuraishi
Mohammed AlQuraishi

Reputation: 1557

Add control dependency after operations are created?

Is it possible to create a control dependency between two ops after they've both been created? I realize that with tf.control_dependencies it's possible to make one op wait on another before executing, but the dependent op has to be created within the tf.control_dependencies context. I'd like to construct the two ops independently first and then add the dependency.

Upvotes: 5

Views: 3611

Answers (2)

Adam Kosiorek
Adam Kosiorek

Reputation: 1528

While not possible in general, you can slightly modify an existing op to enforce dependencies. For example, you can add zero to a tensor: def first_before_second_num(op_first, op_second): with tf.control_dependencies([op_first]): op_second += 0 return op_second

If you have a NoOp like a train_step, then you might want to group it with another NoOp: def first_before_second_noop(op_first, op_second): with tf.control_dependencies([op_first]): op_second tf.group(op_second, tf.no_op()) return op_second

Upvotes: 0

mrry
mrry

Reputation: 126154

This might be a disappointing answer, but it is not possible to add a control dependency (or any other input) to a TensorFlow Operation after it has been created. Tensors and operations are immutable once they have been created.

One possibility would be to clone the op that should run second, in the appropriate control dependencies context. Let's say that you have two Operation objects, op_first and op_second, and you want op_first to run before op_second:

def first_before_second(op_first, op_second):
    """Sequence `op_first` before `op_second`.

    Given two operations, returns a pair of operations with the same behavior
    except that the first returned operation will execute before the second
    returned operation.
    """
    with tf.control_dependencies([op_first]):
        g = tf.get_default_graph()
        op_second_clone = g.create_op(op_second.type,
                                      [in_t for in_t in op_second.inputs],
                                      [out_t.dtype for out_t in op_second.outputs],
                                      attrs=op_second.node_def.attr,
                                      op_def=op_second.op_def)

    return op_first, op_second_clone

A similar modification could make this deal with Tensor objects as well, but that's left as an exercise for the reader.

Upvotes: 8

Related Questions