CODEWITHSUNDEEP
image-processingtensorflowartificial-intelligencedeep-learningmnist
Omar Shehab
Omar Shehab

Reputation: 1094

Understanding the learned weight for MNIST in TensorFlow

I am trying to understand the TensorFlow tutorial which uses MNIST.

I would like to quote the following section:

To tally up the evidence that a given image is in a particular class, we do a weighted sum of the pixel intensities. The weight is negative if that pixel having a high intensity is evidence against the image being in that class, and positive if it is evidence in favor.

The following diagram shows the weights one model learned for each of these classes. Red represents negative weights, while blue represents positive weights.

enter image description here

My question:

Let's take the case of the class 0. So, what does the blue ring mean? The tutorial says blue means 'positive if it is evidence in favor'. What image are we talking about? All or a particular one from the training set? Moreover, what does the red blob in the center mean? Is this a given image? I don't understand what kind of code would generate this image for 0.

Upvotes: 1

Views: 648

Answers (1)

Steven
Steven

Reputation: 5162

If you read further in the tutorial you can see that the equation is written out in matrix format. 

y (the label) = softmax (W^T x + b)

W represents a matrix of weights applied to each pixel, and b represents a number we add to each of the W^T x so it is an elementwise sum. So once we have a trained model W and b are fixed.

We can them solve for what kind of input (x) will give us the highest value for a specific label. So let y = [1,0,0,0,0,0,0,0,0,0,0] (i.e. the label representing the number 0).

softmax^(-1)(y) = W^T x + b

softmax^(-1)(y) - b =  W^T x

W^-T(softmax^(-1)(y) - b) = x

where softmax^(-1) represents the inverse of the softmax function.

This x is the input that would give us the highest value for a particular class in this case 0. We can repeat the process for the other number by changing the vector y appropriately.

We can also plot this x (as it has the same dimensions as our input image and as is shown in your question). We plot it such that negative values are red, values close to 0 are black and positive values are blue.

Upvotes: 2

Related Questions