CODEWITHSUNDEEP
pythonmachine-learningneural-networkpytorch
Matt
Matt

Reputation: 1839

PyTorch Softmax Dimensions error

I'm attempting to write a simple NN module, with 2 layers, first layer ReLU activation, output softmax with 3 classes (one-hot encoded). It seems theres something wrong with the way I'm using the softmax function, but I'm not sure what's going on.

X is 178x13 Y is 178x3

Dataset I'm using is fairly simple, and can be found here.

I keep getting the error:

RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 3) .

.

import pandas as pd
import numpy as np
import torch
from torch.autograd import Variable
from sklearn.preprocessing import LabelBinarizer

# Read in dataset, specifying that this set didn't come with column headers
x = pd.read_csv('Datasets/wine.data', header=None)

# Rename columns
x.columns = ['Class', 'A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A12', 'A13']

y = x[['Class']].values

#turn class labels into one-hot encoding
one_hot = LabelBinarizer()
y = Variable(torch.from_numpy(one_hot.fit_transform(y)), )

x = Variable(torch.from_numpy(x.iloc[:, 1:14].values).float())


N, D_in, H, D_out = y.shape[0], x.shape[1], 20, 3

# Implement neural net with nn module

model = torch.nn.Sequential(
    torch.nn.Linear(D_in, H),
    torch.nn.ReLU(),
    torch.nn.Linear(H, D_out),
    torch.nn.LogSoftmax(dim=3)
)

loss_fn = torch.nn.NLLLoss

learning_rate = 1e-4
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

for t in range(500):
    y_pred = model(x)

    loss = loss_fn(y_pred, y)
    print("Iteration: %d | Loss: %.3f" % (t, loss))

    optimizer.zero_grad()

    loss.backward()

    optimizer.step()

Upvotes: 2

Views: 4591

Answers (2)

Matt
Matt

Reputation: 1839

This was a problem because for NLLLoss:

The target that this loss expects is a class index (0 to N-1, where N = number of classes)

And I had been trying to give it the one-hot encoded vector. I solved my issue by doing:

loss = loss_fn(y_pred, torch.max(y, 1)[1])

Where torch.max found the maximum values and their respective index.

Upvotes: 1

mbpaulus
mbpaulus

Reputation: 7691

It looks to me like you have misunderstood the argument dim of LogSoftmax. From the documentation,

dim (int) – A dimension along which Softmax will be computed (so every slice along dim will sum to 1).

Now, after you pass your input through your two linear layers, the tensor you get and to which you apply LogSoftmax has dimensions 178 x 3. Clearly, dim = 3 is not available as your tensor only has two dimensions. Instead, try dim=1for summing across columns.

Upvotes: 1

Related Questions