CODEWITHSUNDEEP
neural-networkdeep-learningtorch
user3853033
user3853033

Reputation: 55

Using Torch's ClassNLLCriterion

I am currently using Torch and just trying to get a simple neural network program running. Each one of my inputs has 3 attributes and the output is supposed to be a classification between the numbers 1 and 7. I've extracted my data from a CSV file and have put it into 2 Tensors (1 with the inputs and 1 with the outputs). The data is in this format. 

  **Data**
  1914  1993  2386
  1909  1990  2300 
  .....
  1912  1989  2200
  [torch.DoubleTensor of size 99999x3]


  **Class**
  1
  1
  2 
  .....
  7
  [torch.DoubleTensor of size 99999]

For the model I'm using to train the network, I simply have 

 model = nn.Sequential()
 model:add(nn.Linear(3, 7))
 model:add(nn.LogSoftMax())
 criterion = nn.ClassNLLCriterion()

And this is the code I have to train the network

for int i = 1, 10 do
     prediction = model:forward(data)
     loss = criterion:forward(prediction, class)
     model:zeroGradParameters()
     grad = criterion:backward(prediction, class)
     model:backward(data, grad)
     model:updateParameters(.1)
 end

In my test data tensor, I have formatted it in the same way as I formatted the test data (Tensor of 99999x3). I want the program to give me a prediction of what the classification would be when I run this line. 

 print (model:forward(test_data))

However, I am getting negative numbers (which shouldn't happen with ClassNLLCriterion?) and the sums of the probabilities are not adding to 0. My doubt is that I have either not formatted the data correctly or that I wasn't able to perform the training process correctly. If anyone could help me figure out what the issue is, I'd be very grateful.

Thank you!

Upvotes: 0

Views: 392

Answers (2)

Manuel Lagunas
Manuel Lagunas

Reputation: 2751

The reason why you cannot see the prediction yields on the layer model:add(nn.LogSoftMax()) which implements the log function, that's why you have negative values (they are not probabilities). As an example, to get the probabilities back you should do:

model = nn.Sequential()
model:add(nn.Linear(3, 7));
model:add(nn.LogSoftMax());
criterion = nn.ClassNLLCriterion();
data = torch.Tensor{1914,  1993 , 2386}
print (model:forward(data):exp())

>>  0.0000
  0.0000
  1.0000
  0.0000
  0.0000
  0.0000
  0.0000 [torch.DoubleTensor of size 7]

Sorry for the late answer.

Upvotes: 1

stormy
stormy

Reputation: 1

Here is what I currently use which maybe the wrong way of using classnllcriterion but atleast it will get you somewhere to understanding it.

Make the targets to be either 

(7,1,1,1,1,1,1) <--First class representation
.......
(1,1,1,1,1,1,7) <--Last class representation

or 

 (1,1,1,1,1,1,1) <--First class representation
 .......
 (7,7,7,7,7,7,7) <--Last class representation

I figured it's much easier to train the last representation as targets, but I have a feeling we should use the first one instead.

EDIT: I've just found out that classnllcriterion only accepts a scalars as targets, hence using the above is wrong!

You should instead use either 1 .. 7 as target values, either just 1 or just 7.

That's

Upvotes: 0

Related Questions