CODEWITHSUNDEEP
variablestensorflowinitialization
chanwcom
chanwcom

Reputation: 4680

Difference between initializing a Tensorflow Variable using np.zeros and using tf.zeros

Are there some differences between initializing a tensorflow Variable using np.zeros and using tf.zeros?

For example, if we take a look at the MNIST softmax tutorial ( https://github.com/tensorflow/tensorflow/blob/r1.1/tensorflow/examples/tutorials/mnist/mnist_softmax.py), the variables W and b are initialized in the following way.

  W = tf.Variable(tf.zeros([784, 10]))
  b = tf.Variable(tf.zeros([10]))

Instead of initializing using tf.zeros, the code still works fine if we use np.zeros as follows.

  W = tf.Variable(np.zeros([784, 10], dtype=np.float32))
  b = tf.Variable(np.zeros([10], dtype=np.float32))

So, I think there might be no difference. But, I tried the following code segment assuming that the following a and b variables are the same.

a = tf.Variable(tf.zeros((3, 2)), tf.float32)
b = tf.Variable(np.zeros((3, 2)), tf.float32)
sess = tf.InteractiveSession()
init = tf.global_variables_initializer()
sess.run(init)
a.eval()

The result of a.eval() is as follows:

array([[ 0.,  0.],
       [ 0.,  0.],
       [ 0.,  0.]], dtype=float32)

But the result of b.eval() is as follows:

array([[ 0.,  0.],
       [ 0.,  0.],
       [ 0.,  0.]])

So, even though the values are the same, there is a difference in dtype. Could you please explain to me why this difference exists?

Upvotes: 2

Views: 2516

Answers (1)

Salvador Dali
Salvador Dali

Reputation: 222869

For most experiments it almost does not matter. You can also provide python list of lists [[0, 0, ...], ...]. The difference you see in your eval is because tf.zeros by default uses float32. On the contrary np.zeros by default uses float64. Change one or another and you will not see a difference.

In production code it is better to use tf. functions for initialization to reduce the overhead.

Upvotes: 2

Related Questions