CODEWITHSUNDEEP
tensorflow
platinor
platinor

Reputation: 857

In tensorflow what is the difference between tf.add and operator (+)?

In tensorflow tutorials, I see both codes like tf.add(tf.matmul(X, W), b) and tf.matmul(X, W) + b, what is the difference between using the math function tf.add(), tf.assign(), etc and the operators + and =, etc, in precision or other aspects?

Upvotes: 53

Views: 30934

Answers (3)

Aditya Shrivastava
Aditya Shrivastava

Reputation: 277

a = [1,1,1,1]
b = [1,1,1,1]
w = tf.add(a, b)


with tf.Session() as sess:
    p = sess.run(w)
    print(p)

a+b

Now, the value of p printed will be [2,2,2,2] and simple a+b printed will be [1,1,1,1,1,1,1,1].

Upvotes: -4

Salvador Dali
Salvador Dali

Reputation: 222761

Yaroslav nicely explained that there is no real difference. I will just add when using tf.add is beneficial.

tf.add has one important parameter which is name. It allows you to name the operation in a graph which will be visible in tensorboard. So my rule of thumb, if it will be beneficial to name an operation in tensorboard, I use tf. equivalent, otherwise I go for brevity and use overloaded version.

Upvotes: 11

Yaroslav Bulatov
Yaroslav Bulatov

Reputation: 57953

There's no difference in precision between a+b and tf.add(a, b). The former translates to a.__add__(b) which gets mapped to tf.add by means of following line in math_ops.py

_OverrideBinaryOperatorHelper(gen_math_ops.add, "add")

The only difference is that node name in the underlying Graph is add instead of Add. You can generally compare things by looking at the underlying Graph representation like this

tf.reset_default_graph()
dtype = tf.int32
a = tf.placeholder(dtype)
b = tf.placeholder(dtype)
c = a+b
print(tf.get_default_graph().as_graph_def())

You could also see this directly by inspecting the __add__ method. There's an extra level of indirection because it's a closure, but you can get the underlying function as follows

real_function = tf.Tensor.__add__.im_func.func_closure[0].cell_contents
print(real_function.__module__ + "." + real_function.__name__)
print(tf.add.__module__ + "." + tf.add.__name__)

And you'll see output below which means that they call same underlying function

tensorflow.python.ops.gen_math_ops.add
tensorflow.python.ops.gen_math_ops.add

You can see from tf.Tensor.OVERLOADABLE_OPERATORS that following Python special methods are potentially overloaded by appropriate TensorFlow versions

{'__abs__',
 '__add__',
 '__and__',
 '__div__',
 '__floordiv__',
 '__ge__',
 '__getitem__',
 '__gt__',
 '__invert__',
 '__le__',
 '__lt__',
 '__mod__',
 '__mul__',
 '__neg__',
 '__or__',
 '__pow__',
 '__radd__',
 '__rand__',
 '__rdiv__',
 '__rfloordiv__',
 '__rmod__',
 '__rmul__',
 '__ror__',
 '__rpow__',
 '__rsub__',
 '__rtruediv__',
 '__rxor__',
 '__sub__',
 '__truediv__',
 '__xor__'}

Those methods are described in Python reference 3.3.7: emulating numeric types. Note that Python data model does not provide a way to overload assignment operator = so assignment always uses native Python implementation.

Upvotes: 66

Related Questions