CODEWITHSUNDEEP
pythonmachine-learningdeep-learningneural-networkpytorch
Matthew Jacobsen
Matthew Jacobsen

Reputation: 43

Pytorch ConvNet loss remains unchanged and only one class is predicted

My ConvNet is only predicting a single class and the loss remains unchanged.

I have tried the following:

I am unsure of where to go from here. It seems no matter what I try the Neural Net always predicts the same class (I have been able to get it to predict the other class by throwing the weights very out of proportion)

Below is the output from running my program. It should have all the relevant information in it to come up with some ideas as to how to fix it. If you need to see some of the source code or are curious about what the dataset looks like, please ask.

Any help is greatly appreciated. I have been stumped on this issue for quite a while now. Thank you!

Train dataset length: 27569
Test dataset length: 4866

Image preprocessing: 
None
Input dimensions: 28 X 28
Output dimension: 2

Model: Simple
NeuralNetwork(
  (flatten): Flatten(start_dim=1, end_dim=-1)
  (linear_relu_stack): Sequential(
    (0): Linear(in_features=784, out_features=512, bias=True)
    (1): ReLU()
    (2): Linear(in_features=512, out_features=512, bias=True)
    (3): ReLU()
    (4): Linear(in_features=512, out_features=2, bias=True)
    (5): ReLU()
  )
)

Optimizer: Adam
Learning rate: 0.0001

Loss function: CEL
class weights: tensor([0.3481, 0.6519], device='cuda:0')
Multiplicative factor of learning rate decay: 0.0005

Train Epoch: 1 [0/27569 (0%)]   Loss: 3785.907959
Train Epoch: 1 [6400/27569 (23%)]   Loss: 0.693147
Train Epoch: 1 [12800/27569 (46%)]  Loss: 0.693147
Train Epoch: 1 [19200/27569 (70%)]  Loss: 0.693147
Train Epoch: 1 [25600/27569 (93%)]  Loss: 0.693147

Test set: Average loss: 0.0110, Accuracy: 3172/4866 (65%)

actual count: [3172, 1694]
predicted count: [4866, 0]

Upvotes: 0

Views: 159

Answers (1)

Berriel
Berriel

Reputation: 13601

It is very uncommon to have a ReLU after the last Linear layer (where the logits come from). Consider removing it.

In addition, maybe your learning rate is too high. You could try tweaking it a little bit. Check if the loss decreases smoothly between the iterations (which is ideal in most cases), otherwise decrease it

Upvotes: 2

Related Questions