CODEWITHSUNDEEP
pythonmachine-learningtensorflow
YOON
YOON

Reputation: 47

Tensorflow - Does Weight value changed in tf.nn.conv2D()?

While I'm studying about Neural Network with tensorflow, I got a some question about tf.nn.conv2D(x, W, strides=[1, 1, 1, 1], padding='SAME')

When I input image value x, and weight value W(initailzed by tf.truncated_normal(shape, stddev=0.1)), I understand it will return some value, which is the result of tf.nn.conv2D().

But my question is, when tf.nn.conv2D() is called, Does It change weight value?

If it changes the value of Weight, How does it works? In fact, When I print Weight value, it changes. But I don't know why... My assumption is that value W is some kind of call-by-reference, so while computing tf.nn.conv2D(), the value W is changed. Is it right?

Upvotes: 0

Views: 1189

Answers (1)

Desh Raj
Desh Raj

Reputation: 113

The Tensorflow code flow is not like your conventional programming language. First, a graph is created from the code (which can be visualized using Tensorboard, and then update rules are computed using backpropagation which has been implemented internally.

When you write:

h = tf.nn.conv2D(x, W, strides=[1, 1, 1, 1], padding='SAME')

it creates a convolutional layer in your neural network, which performs convolution (http://cs231n.github.io/convolutional-networks/) on your input matrix and outputs the result in h. Now, the whole purpose of performing such convolutions is to identify some local patterns such as vertical or horizontal edges in the image. For example, a weight matrix W such as

W = [[0,1,0],[0,1,0],[0,1,0]]

would identify vertical edges in the image. However, since W has been initialized randomly here

W = tf.Variable(tf.truncated_normal(shape, stddev=0.1)))

would not be able to find any pattern at the outset. This is solved through backpropagation.

When you train your neural network on a labeled data, at each step the matrix W gets updated such that the derivative of the error E w.r.t W is reduced. You cannot see it happening in your code because the backpropagation is implemented internally in Tensorflow and you only need to write code for the forward pass. If you defined W as

W = tf.Variable(tf.truncated_normal(shape, stddev=0.1)),trainable=False)

it won't be updated, but then the entire purpose of training the parameters would be defeated.

I suggest you to go through http://neuralnetworksanddeeplearning.com to understand how neural networks work, before you proceed with Tensorflow.

Upvotes: 1

Related Questions