CODEWITHSUNDEEP
pythonpycharmpytorch
Smighty
Smighty

Reputation: 85

PyTorch: Expected input batch_size (12) to match target batch_size (64)

I tried out PyTorch and wanted to write a program for MNIST. But, I got the error message:

Expected input batch_size (12) to match target batch_size (64)

I searched for a solution but I don't understand what's wrong with my code.

#kwargs is empty because I don't use cuda
kwargs = {}
train_data = torch.utils.data.DataLoader(
    datasets.MNIST('data', train=True, download=True,
                    transform=transforms.Compose([transforms.ToTensor(),
                    transforms.Normalize((0.1307,),(0.3081,))])),
    batch_size=64, shuffle=True, **kwargs)

test_data = torch.utils.data.DataLoader(
    datasets.MNIST('data', train=False,
                    transform=transforms.Compose([transforms.ToTensor(),
                    transforms.Normalize((0.1307,),(0.3081,))])),
    batch_size=64, shuffle=True, **kwargs)

class Netz(nn.Module):
    def __init__(self):
        super(Netz, self).__init__()
        self.conv1 = nn.Conv2d(1,10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv_dropout = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 60)
        self.fc2 = nn.Linear(60, 10)

    def forward(self, x):
        x = self.conv1(x)
        x = F.max_pool2d(x, 2)
        x = F.relu(x)
        x = self.conv2(x)
        x = self.conv_dropout(x)
        x = F.max_pool2d(x, 2)
        x = F.relu(x)
        print(x.shape)
        x = x.view(-1, 320)
        x = self.fc1(x)
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return F.log_softmax(x, dim=0)

model = Netz()

optimizer = optim.SGD(model.parameters(), lr=0.1, momentum=0.8)
def train(epoch):
    model.train()

    for batch_id, (data, target) in enumerate(train_data):
        data = Variable(data)
        target = Variable(target)
        optimizer.zero_grad()
        out = model(data)
        print(out.shape)
        criterion = nn.CrossEntropyLoss()
        loss = criterion(out, target)
        loss.backward()
        optimizer.step()
        print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'. format(
            epoch, batch_id * len(data), len(train_data.dataset),
            100. * batch_id / len(train_data), loss.data[0]))

The output should show the epoch and some other information. Actually, I print out the shape of my tensor but I don't know what's wrong. Here is the error message:

/home/michael/Programmierung/Python/PyTorch/venv/bin/python /home/michael/Programmierung/Python/PyTorch/mnist.py
torch.Size([64, 20, 4, 4])
torch.Size([12, 10])
Traceback (most recent call last):
  File "/home/michael/Programmierung/Python/PyTorch/mnist.py", line 69, in <module>
    train(epoch)
  File "/home/michael/Programmierung/Python/PyTorch/mnist.py", line 60, in train
    loss = criterion(out, target)
  File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 942, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 2056, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 1869, in nll_loss
    .format(input.size(0), target.size(0)))
ValueError: Expected input batch_size (12) to match target batch_size (64).

Process finished with exit code 1

Upvotes: 4

Views: 17824

Answers (1)

adeelh
adeelh

Reputation: 526

The error occurs because your model output, out, has shape (12, 10), while your target has a length of 64.

Since you are using a batch size of 64 and predicting the probabilities of 10 classes, you would expect your model output to be of shape (64, 10), so clearly there is something amiss in the forward() method.

Going through it line by line and noting the size of x at every step, we can try to find out what is going wrong:

    ...
    # x.shape = (64, 20, 4, 4) at this point as seen in your print statement
    x = x.view(-1, 320)             # x.shape = (64, 320)
    x = self.fc1(x)                 # x.shape = (64, 60)
    x = x.view(-1, 320)             # x.shape = (12, 320)
    x = F.relu(self.fc1(x))         # x.shape = (12, 60)
    x = self.fc2(x)                 # x.shape = (12, 10)
    return F.log_softmax(x, dim=0)  # x.shape = (12, 10)

What you actually most likely want is:

    ...
    # x.shape = (64, 20, 4, 4) at this point as seen in your print statement
    x = x.view(-1, 320)             # x.shape = (64, 320)
    x = F.relu(self.fc1(x))         # x.shape = (64, 60)
    x = self.fc2(x)                 # x.shape = (64, 10)
    return F.log_softmax(x, dim=1)  # x.shape = (64, 10)

Note: While not related to the error, note also that you want to softmax over dim=1 since that is the dimension that contains the logits for the classes.

Upvotes: 11

Related Questions