CODEWITHSUNDEEP
pythonpandasmachine-learningdeep-learningpytorch
eric424
eric424

Reputation: 3

why is my loss function only returning NaN values?

below is my code

import numpy as np
import torch
from torch.utils import data
import torch.nn as nn
import pandas as pd

# PREPPING DATA FROM CSV FILE
csvFile = pd.read_csv('/Users/ericbeep999/Desktop/Web Development/Projects/Python/pytorch/3. Linear Regression/weather.csv')
labels, features = csvFile.iloc[:, 4], csvFile.iloc[:, 5]
#labels - min temp
#features - max temp
labels = torch.tensor(labels, dtype=torch.float32).reshape(-1, 1)
features = torch.tensor(features, dtype=torch.float32).reshape(-1,1)





# READING DATASET 
def load_array(data_arrays, batch_size, is_train = True):
    dataset = data.TensorDataset(*data_arrays)
    return data.DataLoader(dataset, batch_size, shuffle= is_train)
batch_size = 20
data_set = load_array((features, labels), batch_size)


#DEFININING MODEL AND PARAMETERS
model = nn.Sequential(nn.Linear(1, 1))
model[0].weight.data.normal_(0, 0.1)
model[0].bias.data.fill_(0)

#DEFINING LOSS FUNCTION AND OPTIMIZATION ALGORITHMN 

lossFunc = nn.MSELoss()
learning_rate = 0.01
gradient = torch.optim.SGD(model.parameters(), learning_rate)


#TRAINING MODEL

num_epochs = 100
for epoch in range(num_epochs):
    for X, Y in data_set:

        loss = lossFunc(model(X), Y)
        gradient.zero_grad()
        loss.backward()
        gradient.step()
    loss = lossFunc(model(features), labels)
    print(f'epoch: {epoch + 1}, loss: {loss}')

print(f"{model[0].weight.data}, {model[0].bias.data}")

the csv file I am importing the data from can be found at https://www.kaggle.com/datasets/smid80/weatherww2?datasetId=3759&searchQuery=pytorch

My labels are the min temperature and my features are the max temperature

whenever I run the code, the only thing that prints is

epoch: 1, loss: nan
epoch: 2, loss: nan
epoch: 3, loss: nan
epoch: 4, loss: nan
epoch: 5, loss: nan
epoch: 6, loss: nan
epoch: 7, loss: nan
epoch: 8, loss: nan
epoch: 9, loss: nan
epoch: 10, loss: nan

i don't really understand why it is only printing NaN

Upvotes: 0

Views: 441

Answers (1)

adamconkey
adamconkey

Reputation: 4745

I changed your learning rate to 0.001 and it runs without giving NaNs (albeit not learning anything since predicting min temperature from max temperature in that data may not be easily learned). My guess is the issue is with the scale of your input/output data, i.e. they're in the range of something like 0-40 which isn't great for neural networks - it can cause the learning to go unstable more easily. I would suggest you first scale your inputs/outputs to be in a range of [0, 1] or [-1, 1]. I'll direct you to this blog for details on achieving that, they also discuss in more detail why scaling the data is important for learning with neural networks.

Upvotes: 1

Related Questions