CODEWITHSUNDEEP
pythondeep-learningpytorchconv-neural-networkcross-entropy
Helloworld8688
Helloworld8688

Reputation: 111

RuntimeError: 0D or 1D target tensor expected, multi-target not supported I was training a deep learning model but I am getting this issue

*My Training Model*
def train(model,criterion,optimizer,iters):
    epoch = iters
    train_loss = []
    validaion_loss = []
    train_acc = []
    validation_acc = []
    states = ['Train','Valid']
    for epoch in range(epochs):
        print("epoch : {}/{}".format(epoch+1,epochs))
        for phase in states:
            if phase == 'Train':
                model.train() *training the data if phase is train*
                dataload = train_data_loader
            else:
                model.eval()
                dataload = valid_data_loader
        
            run_loss,run_acc = 0,0 *creating variables to calculate loss and acc*
            for data in dataload:
                inputs,labels = data
                inputs = inputs.to(device)
                labels = labels.to(device)
            
                labels = labels.byte()
                optimizer.zero_grad() #Using the optimizer
            
                with torch.set_grad_enabled(phase == 'Train'):
                    outputs = model(inputs)
                    loss = criterion(outputs,labels.unsqueeze(1).float())
                
                    predict = outputs>=0.5
                    if phase == 'Train':
                        loss.backward() #backward propagation
                        optimizer.step()
                
                    acc = torch.sum(predict == labels.unsqueeze(1))
                run_loss+=loss.item()
                run_acc+=acc.item()/len(labels)
            if phase == 'Train': #calulating train loss and accucracy
                epoch_loss = run_loss/len(train_data_loader)
                train_loss.append(epoch_loss)
                epoch_acc = run_acc/len(train_data_loader)
                train_acc.append(epoch_acc)
            else: #training validation loss and accuracy
                epoch_loss = run_loss/len(valid_data_loader)
                validaion_loss.append(epoch_loss)
                epoch_acc = run_acc/len(valid_data_loader)
                validation_acc.append(epoch_acc)
        
            print("{}, loss :{},accuracy:{}".format(phase,epoch_loss,epoch_acc))
    
    history = {'Train_loss':train_loss,'Train_accuracy':train_acc,
               'Validation_loss':validaion_loss,'Validation_Accuracy':validation_acc}
    return model,history[enter image description here][1]

I was experiencing the error as 0D or 1D target tensor expected, multi-target not supported could you please help in rectifying the code which is described above. Referred the previous related articles but unable to get the desired result. What are the code snippets I had to change so that my model will run successfully. Any suggestions are mostly welcome. Thanks in Advance.

Upvotes: 8

Views: 38725

Answers (3)

zia badar
zia badar

Reputation: 11

For me the issue was the one_hot not getting used with num_classes parameter because without it, chances are your batch might not contain elements of a particular class which makes the dimension of one_hot encoding different. Use num_classes parameter in one_hot to fix the issue.

# for example if I have 4 classes with labels[0, 1, 2, 3], I expect one_hot to return Nx4 tensor.

# case where size(N, 4) is correct because last class is part of batch
F.one_hot(torch.tensor([3]))
tensor([[0, 0, 0, 1]])

# case where size(N, 3) is incorrect because last class is not part of batch
F.one_hot(torch.tensor([0, 1, 2]))
tensor([[1, 0, 0],
        [0, 1, 0],
        [0, 0, 1]])

# specify num_classes parameter to get the correct size tensor
F.one_hot(torch.tensor([0, 1, 2]), num_classes = 4)
tensor([[1, 0, 0, 0],
        [0, 1, 0, 0],
        [0, 0, 1, 0]])

This is a higher chance of this happening at the end of dataloader, because drop_last=False causes the batch_size to be smaller than the dataloader batch_size.

Upvotes: 1

Mujeeb Ishaque
Mujeeb Ishaque

Reputation: 2711

This issue can also be due to loss function. Try using alternative loss functions that can deal with multi-target tensor. I used nn.MSELoss() and the error went away.

Upvotes: 3

Hamzah Al-Qadasi
Hamzah Al-Qadasi

Reputation: 9786

Your problem is that labels have the correct shape to calculate the loss. When you add .unsqueeze(1) to labels you made your labels with this shape [32,1] which is not consistent to the requirment to calcualte the loss.

To fix the problem, you only need to remove .unsqueeze(1) for labels.

If you read the documentation of CrossEntropLoss, the arguments:

  • Input should be in (N,C) shape which is outputs in your case and [32,3].
  • Target should be in N shape which is labels in your case and should be [32]. Therefore, the loss function expects labels to be in 1D target not multi-target.

Upvotes: 14

Related Questions