CODEWITHSUNDEEP
machine-learningneural-networkpytorch
Sanskar Kumar
Sanskar Kumar

Reputation: 473

Pytorch showing the error: 'NoneType' object has no attribute 'zero_'

I am using Python 3.8 and VSCode.

I tried to create a basic Neural Network without activations and biases but because of the error, I'm not able to update the gradients of the weights.

Matrix Details:

Layer Shape: (1, No. of Neurons)

Weight Layer Shape: (No. of Neurons in the previous layer, No. of Neurons in the next layer)

Here's my code:

# sample neural network

# importing libraries
import torch
import numpy as np
torch.manual_seed(0)

# hyperparameters
epochs = 100
lr = 0.01 # learning rate

# data
X_train = torch.tensor([[1]], dtype = torch.float)
y_train = torch.tensor([[2]], dtype = torch.float)

'''
Network Architecture:
    1 neuron in the first layer
    4 neurons in the second layer
    4 neurons in the third layer
    1 neuron in the last layer

    * I haven't added bias and activation.
'''

# initializing the weights
weights = []
weights.append(torch.rand((1, 4), requires_grad = True))
weights.append(torch.rand((4, 4), requires_grad = True))
weights.append(torch.rand((4, 1), requires_grad = True))

# calculating y_pred
y_pred = torch.matmul(torch.matmul(torch.matmul(X_train, weights[0]), weights[1]), weights[2])

# calculating loss
loss = (y_pred - y_train)**2

# calculating the partial derivatives
loss.backward()

# updating the weights and zeroing the gradients
with torch.no_grad():
    for i in range(len(weights)):
        weights[i] = weights[i] - weights[i].grad
        weights[i].grad.zero_()

It it showing the error:

File "test3.py", line 43, in <module>
weights[i].grad.zero_()
AttributeError: 'NoneType' object has no attribute 'zero_'

I don't understand why it is showing this error. Can someone please explain?

Upvotes: 0

Views: 3525

Answers (1)

Alex Metsai
Alex Metsai

Reputation: 1950

Your model doesn't have any trainable parameters for the grad to be calculated. Use torch's Parameter. See this link for creating a module with learnable parameters.

  torch.nn.parameter.Parameter
        A kind of Tensor that is to be considered a module parameter.

Upvotes: 1

Related Questions