CODEWITHSUNDEEP
pythonpytorch
An Ignorant Wanderer
An Ignorant Wanderer

Reputation: 1632

Understanding the parameters of a simple neural network in Pytorch

I have defined a simple NN as follows in Pytorch: 

my_model = nn.Sequential(
            nn.Linear(1, 5),
            nn.Tanh(),
            nn.Linear(5, 1))

I then iterate through the parameters and check their sizes:

[parameter.shape for parameter in my_model.parameters()]

I get:

[torch.Size([5, 1]), torch.Size([5]), torch.Size([1, 5]), torch.Size([1])]

I'm confused as to why the last Size is 1. Shouldn't there be 5 bias values as well, going out from the hidden layer into the output layer?

Upvotes: 3

Views: 443

Answers (2)

yanarp
yanarp

Reputation: 175

The number of biases in any layer depends on the number of output nodes of that layer. Since your final layer has one output node, it only has one bias. It applies to Convolutional layers as well, in that case, number biases is number of output channels you define.

Upvotes: 0

Sraw
Sraw

Reputation: 20234

Um, so it looks like you are defining a two layers network.

The input size is [1], and the hidden layer size is [5], and next, the hidden layer is connected to the output layer, the output layer size is [1].

It should look like:

       *   <- input
   / / | \ \
   * * * * *  <- hidden layer
   \ \ | / /
       *   <- output

So for the hidden layer, there are 5 values as bias, for the output layer, there is only one value as bias. Sounds reasonable?

Upvotes: 2

Related Questions