CODEWITHSUNDEEP
pythonneural-networktheano
amartya18x
amartya18x

Reputation: 35

Theano batch learning in MLP, shape of bias

I am confused about this. Say I have a weight matrix W(10,20). This expects a 20 dimensional input vector(say X). So, dot(W,X) will be (10,1) and to this I can add a 10 dimensional bias vector.

Now I want to do batch learning. Is it possible to multiplt a batch of input vectors(30 vectors), so that X is now(20,30) and then dot(W,X) will be (10,30). But what about bias. how should I adjust bias to this ?

Upvotes: 2

Views: 201

Answers (1)

Konstantin Schubert
Konstantin Schubert

Reputation: 3356

I assume the bias is the same for all 30 input vectors. Let me know if this is not the case.

Think of your (10,30) matrix as a collection of 30 10-dimensional result vectors. They are independent, and they are just arranged into a matrix for easier handling.

If you want to add the 10-dimensional bias vector to your 30 result vectors, all you have to do is to copy your 10-dimensional bias vector 30 times into a (10,30) bias matrix. Then, adding the matrices will be the same as adding the bias vector to each of your 30 10-dimensional result vectos

You can obtain this by multiplying 

<vertical bias vector>  <outer/tensor product>  <horizontal vector, filled with 30 ones>

Upvotes: 2

Related Questions