CODEWITHSUNDEEP
pytorch
Chandra
Chandra

Reputation: 524

Linear regression with pytorch

I tried to run linear regression on ForestFires dataset. Dataset is available on Kaggle and gist of my attempt is here: https://gist.github.com/Chandrak1907/747b1a6045bb64898d5f9140f4cf9a37

I am facing two problems:

  1. Output from prediction is of shape 32x1 and target data shape is 32.

input and target shapes do not match: input [32 x 1], target [32]¶

Using view I reshaped predictions tensor.

y_pred = y_pred.view(inputs.shape[0])

Why there is a mismatch in shapes of predicted tensor and actual tensor?

  1. SGD in pytorch never converges. I tried to compute MSE manually using

print(torch.mean((y_pred - labels)**2))

This value does not match

loss = criterion(y_pred,labels)

Can someone highlight where is the mistake in my code?

Thank you.

Upvotes: 1

Views: 1422

Answers (1)

Viet Phan
Viet Phan

Reputation: 2119

Problem 1

This is reference about MSELoss from Pytorch docs: https://pytorch.org/docs/stable/nn.html#torch.nn.MSELoss

Shape:
 - Input: (N,∗) where * means, any number of additional dimensions
 - Target: (N,∗), same shape as the input

So, you need to expand dims of labels: (32) -> (32,1), by using: torch.unsqueeze(labels, 1) or labels.view(-1,1)

https://pytorch.org/docs/stable/torch.html#torch.unsqueeze

torch.unsqueeze(input, dim, out=None) → Tensor

Returns a new tensor with a dimension of size one inserted at the specified position.

The returned tensor shares the same underlying data with this tensor.

Problem 2

After reviewing your code, I realized that you have added size_average param to MSELoss: 

criterion = torch.nn.MSELoss(size_average=False)

size_average (bool, optional) – Deprecated (see reduction). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average is set to False, the losses are instead summed for each minibatch. Ignored when reduce is False. Default: True

That's why 2 computed values not matched. This is sample code:

import torch
import torch.nn as nn

loss1 = nn.MSELoss()
loss2 = nn.MSELoss(size_average=False)
inputs = torch.randn(32, 1, requires_grad=True)
targets = torch.randn(32, 1)

output1 = loss1(inputs, targets)
output2 = loss2(inputs, targets)
output3 = torch.mean((inputs - targets) ** 2)

print(output1)  # tensor(1.0907)
print(output2)  # tensor(34.9021)
print(output3)  # tensor(1.0907)

Upvotes: 2

Related Questions