CODEWITHSUNDEEP
pythonpytorchgradientbackpropagationautograd
user118967
user118967

Reputation: 5722

Why doesn't torch.autograd compute the gradient in this case?

Why doesn't torch.autograd compute the gradient in this case?

import torch
x = torch.tensor([1., 2., ], requires_grad=True)
y = torch.tensor([x[0] + x[1], x[1] - x[0], ], requires_grad=True)
z = y[0] + y[1]
z.backward()
x.grad

Output is a blank line (None). The same occurs for x[0].grad. Why?

PS: In retrospect, I realize the motivation for making y a tensor with requires_grad was so I could examine its own gradient. I learned that one must use retain_grad for that here: Why does autograd not produce gradient for intermediate variables?

Upvotes: 1

Views: 371

Answers (1)

Sergii Dymchenko
Sergii Dymchenko

Reputation: 7209

When you use torch.tensor for y, it just uses the values of x to initialize the tensor, the gradient chain is lost.

This works:

x = torch.tensor([1., 2., ], requires_grad=True)
y = [x[0] + x[1], x[1] - x[0], ]
z = y[0] + y[1]
z.backward()
x.grad

The result is tensor([0., 2.])

Upvotes: 2

Related Questions