eliss
eliss

Reputation: 89

Difference in autograd.grad for same input

I am trying to calculate the gradient of some points using autograd.grad(). The code is the following:

for batch in data_loader:
  batch_coordinates = batch[0]
  out = model(batch_coordinates.requires_grad_())

  x1, y1 = batch_coordinates[:, 0].reshape(-1, 1), batch_coordinates[:, 1].reshape(-1, 1)

  y = custom_output(out, x1, y1)
  grad_1 = torch.autograd.grad(y.sum(), batch_coordinates, create_graph=True) #calulate gradient
  grad_2 = torch.autograd.grad(y.sum(), (x1, y1), create_graph=True) #calulate gradient

Where batch_coordinates is a torch tensor of dimension torch.Size([n, 2]), with n the number of points; and the model a very simple three-layered NN. custom_output(out, x1, y1) is a function that I need to interpolate the result of the network and takes in input the x,y of the points and the result of the network. My question is: why are grad_1/2 different (accounting for the fact that the grad_2 is a list)?

the output is:

GRAD MINIBATCH COORDINATES: 
(tensor([[-6.5256e-03,  1.5925e-03],
        [-6.1777e-03, -8.8977e-05],
        [-2.5146e-03,  3.1782e-03],
        [-4.3833e-03,  2.2729e-03],
        [-6.6386e-03,  3.9808e-04],
        [-5.9642e-03, -2.2408e-04],
        [ 1.0123e-03,  4.7309e-03],
        [ 3.8920e-02, -1.3823e-01],
        [-6.2750e+00,  1.4732e+00],
        [-5.8813e-03, -2.6810e-04]], grad_fn=<AddBackward0>),)
GRAD X1, Y1: 
(tensor([[ 0.0000e+00],
        [ 0.0000e+00],
        [ 0.0000e+00],
        [ 0.0000e+00],
        [-2.0982e-05],
        [ 0.0000e+00],
        [-1.5622e-04],
        [ 4.5191e-02],
        [-6.2711e+00],
        [ 0.0000e+00]], grad_fn=<AddBackward0>), tensor([[ 0.0000e+00],
        [ 0.0000e+00],
        [ 0.0000e+00],
        [ 0.0000e+00],
        [ 7.6683e-06],
        [ 0.0000e+00],
        [-2.7294e-05],
        [-1.3825e-01],
        [ 1.4730e+00],
        [ 0.0000e+00]], grad_fn=<AddBackward0>))

Upvotes: 1

Views: 39

Answers (0)

Related Questions