CODEWITHSUNDEEP
pytorch
laridzhang
laridzhang

Reputation: 1801

Pytorch 0.4: Should the requires_grad flag of the network input be set to True during training?

The requires_grad flag of input is default False during my last training. I was wondering if I should set it to True.

Upvotes: 2

Views: 426

Answers (1)

MBT
MBT

Reputation: 24129

So generally:

  • For all tensors / weights that you want to be trained the requires_grad flag has to be True.
    This would be the case for your parameters resp. weights and biases. So there you want the flag to be True. But this is already default value for predefined modules like nn.Linear, nn.Embedding. nn.Conv2d etc. So you don't have to change it.

  • For things like inputs the requires_grad flag needs to be False since you don't want to train your input data.

I hope this answers your question, if you want to know something more specific just ask!

Upvotes: 3

Related Questions