CODEWITHSUNDEEP
pythonconv-neural-networkpytorch
Jonathan Bechtel
Jonathan Bechtel

Reputation: 3607

Dimensions Not Matching In PyTorch Linear Layer

Following the training lesson in PyTorch on this page: https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py

It's basically their 'Hello World!' version of an image classifier.

What I'm trying to do is manually code the training steps in the network to make sure I understand each one, but I am currently getting a dimension mismatch in one of my linear layers, which has me stumped. Especially since (AFAIK) I'm recreating the steps in the tutorial exactly.

Anyways........

MY NETWORK:

class net(nn.Module):
def __init__(self):
    super(net, self).__init__()
    self.conv1 = nn.Conv2d(3, 6, 5)
    self.pool  = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc1   = nn.Linear(16*5*5, 120)
    self.fc2   = nn.Linear(120, 84)
    self.fc2   = nn.Linear(84, 10)

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = x.view(-1, 16 * 5 * 5)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)

    return x

net = net()

I believe this is exactly as they have it on their own page.

I'm trying to calculate the following step without a loop:

for epoch in range(2):  # loop over the dataset multiple times

  running_loss = 0.0
  for i, data in enumerate(trainloader, 0):
    # get the inputs; data is a list of [inputs, labels]
    inputs, labels = data

    # zero the parameter gradients
    optimizer.zero_grad()

    # forward + backward + optimize
    outputs = net(inputs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

    # print statistics
    running_loss += loss.item()
    if i % 2000 == 1999:    # print every 2000 mini-batches
        print('[%d, %5d] loss: %.3f' %
              (epoch + 1, i + 1, running_loss / 2000))
        running_loss = 0.0

  print('Finished Training')

What I'm doing is this:

data = enumerate(trainloader)
inputs, labels = next(data)[1]
outputs = net(inputs)

And the last line gives me the following traceback:

RuntimeError                              Traceback (most recent call last)
<ipython-input-285-d4be5abf5bb1> in <module>
----> 1 outputs = net(inputs)

~\Anaconda\lib\site-packages\torch\nn\modules\module.py in __call__(self, 
*input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

<ipython-input-282-a6eca2e3e9db> in forward(self, x)
    14         x = x.view(-1, 16 * 5 * 5)
    15         x = F.relu(self.fc1(x))
---> 16         x = F.relu(self.fc2(x))
    17         x = self.fc3(x)

Which closes out with:

RuntimeError: size mismatch, m1: [4 x 120], m2: [84 x 10] at 
c:\a\w\1\s\tmp_conda_3.7_110206\conda\conda- 
bld\pytorch_1550401474361\work\aten\src\th\generic/THTensorMath.cpp:940

I know this means my dimension values don't match, and I suspect it has to do with the line x = x.view(-1, 16 * 5 * 5) where I go from the convolutional to linear layer, but I have two confusions:

Upvotes: 0

Views: 1838

Answers (2)

Andr&#233; Pacheco
Andr&#233; Pacheco

Reputation: 1885

I don't know what is the size of the images you're using, but it seems that it's affecting the size of the very last feature map, hence, the amount of data you send to the linear model.

Try to check the size doing this: in your forward method:

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    print (x.shape)

You're gonna have something like: torch.Size([32, 16, 4, 4]) Then, you can use x = x.view(-1, 16 * 4 * 4)

Alternatively, you can get these values directly from x.shape[1:].

Upvotes: 1

Anubhav Singh
Anubhav Singh

Reputation: 8699

Actually, there is no self.fc3(x) in __init__() as you have mentioned in forward() function. Try running you code by changing

self.fc2 = nn.Linear(84, 10) in __init__() function to self.fc3 = nn.Linear(84, 10).

Above mistake is the reason why you are getting the error. As you are initializing self.fc2 twice in the above code, see below lines:

self.fc2   = nn.Linear(120, 84)
self.fc2   = nn.Linear(84, 10)

Here, first value of self.fc2 is overriden by later value. So, finally it is initialized with a Linear layer with input channels 84 and output channels 10. Later on, in the forward function you are passing the output channels of x = F.relu(self.fc1(x)), i.e., 120 as an input channels to x = F.relu(self.fc2(x)), which has been changed to 84 because of the above explained reasons, you are getting the error.

Apart from this, I don't think if something wrong with your code.

Upvotes: 2

Related Questions