CODEWITHSUNDEEP
machine-learningneural-networkpytorch
Miles-can
Miles-can

Reputation: 59

Error " 'Softmax' object has no attribute 'log_softmax' " while training Neural Network with PyTorch

I am working on a classifier for MNIST dataset. When I am running the code below, I am getting the error " 'Softmax' object has no attribute 'log_softmax' " at line loss = loss_function(output, y). I have not managed to find a solution to the problem. I will appreciate if you can advise on how the issue can be resolved. Thank you.

import matplotlib.pyplot as plt
import torch 
import torch.nn as nn
import torch.nn.functional as F 
import torch.optim as optim
from torch.utils.data import DataLoader, Dataset, TensorDataset
import torchvision
import torchvision.transforms as transforms
import numpy as np


device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
batch_size = 512

# Image transformations of Torchvision will convert to the images to tensor and normalise with mean and standard deviation
transformer = transforms.Compose([transforms.ToTensor(), transforms.Normalize(mean=(0.1307,), std=(0.3081,))])

data_train = DataLoader(torchvision.datasets.MNIST('Data/data/mnist', download=True, train=True, transform=transformer), 
                batch_size=batch_size, drop_last=False, shuffle=True)
data_test = DataLoader(torchvision.datasets.MNIST('Data/data/mnist', download=True, train=False, transform=transformer), 
                batch_size=batch_size, drop_last=False, shuffle=True)


class neural_nw(nn.Module):
    
    def __init__(self):
        super(neural_nw, self).__init__()
        self.fc1 = nn.Linear(784, 128, True)
        self.fc2 = nn.Linear(128, 128, True)
        self.fc3 = nn.Linear(128, 10, True)
    
    def forward(self, x):
        output = torch.sigmoid(self.fc1(x))
        output = torch.sigmoid(self.fc2(output))
        output = nn.Softmax(self.fc3(output))
        return output
    
MLP = neural_nw()

loss_function = nn.CrossEntropyLoss()
optimiser = optim.Adam(MLP.parameters(), lr = 0.01)

Epochs = 50

for epoch in range(Epochs):
    for X, y in data_train:
        X = X.view(X.shape[0], -1)
        optimiser.zero_grad()
        output = MLP.forward(X)
        loss = loss_function(output, y)
        loss.backward()
        optimiser.step()

Upvotes: 2

Views: 8565

Answers (3)

Abednego
Abednego

Reputation: 1

I think you need to change your nn.softmax to F.softmax why? because the nn.functional contains the softmax operation, and you have imported it already in our code torch.nn.functional as F. So to solve the problem, just make it F.softmax instead of nn.softmax.

Upvotes: 0

ndrwnaguib
ndrwnaguib

Reputation: 6115

nn.Softmax defines a module, nn.Modules are defined as Python classes and have attributes, e.g., a nn.LSTM module will have some internal attributes like self.hidden_size. On the other hand, F.softmax defines the operation and needs all arguments to be passed (including the weights and bias). Implicitly, the modules will usually call their functional counterpart somewhere in the forward method.

This explains why F.softmax instead of nn.Softmax resolves your issue.

Upvotes: 2

Miles-can
Miles-can

Reputation: 59

Replacing output = nn.Softmax(self.fc3(output)) with output = F.softmax(self.fc3(output)) seems to solve the issue.

Upvotes: 0

Related Questions