CODEWITHSUNDEEP
pythonpytorchcomputer-visiongradient-descent
Craving_gold
Craving_gold

Reputation: 189

How can I get the gradients of two losses in pytorch

I am trying to get the gradients of two losses in the following code snippet but all I get is None (AttributeError: ‘NoneType’ object has no attribute ‘data’)

img = img.to(device)
#img.requires_grad = True
input = model(img)
input_prediction = input.max(1, keepdim=True)[1]

btarget = torch.tensor(2).unsqueeze(0).to(device)
x_prime.requires_grad = True
x_prime_output = model(x_prime)
x_prime_pred =  x_prime_output.max(1, keepdim=True)[1]

l_target_loss = F.nll_loss(x_prime_output, btarget)
# model.zero_grad()
l_target_loss.retain_grad()
l_target_loss.backward(retain_graph = True)
target_grad = l_target_loss.grad.data

l_argmax_loss = F.nll_loss(x_prime_output, input_prediction.squeeze(0))
l_argmax_loss.retain_grad()
l_argmax_loss.backward()
l_argmax_grad = l_argmax_loss.grad.data

when I tried getting target_grad and l_argmax_grad, I get None.

Edit: I am actually trying to get the gradient of the l_target_loss w.r.t to the input x and the gradient of the l_argmax_loss w.r.t to the input x as well

Upvotes: 1

Views: 1334

Answers (1)

Ivan
Ivan

Reputation: 40618

You need to retain the gradient on that tensor with retain_grad, by default it is not cached in memory:

>>> l_target_loss.retain_grad()
>>> l_target_loss.backward(retain_graph=True)

Upvotes: 1

Related Questions