kobako
Reputation: 673
In pytorch, which part cost most time? Loss calculation or backward+step?
In pytorch, which part cost most of time?
- Loss calculation(no backward, just calculating)
- backward
- step Please explain a bit if you can.
Upvotes: 0
Views: 109
Answers (1)
Prajot Kuvalekar
Reputation: 6618
Not just in Pytorch, but in all framework, backward_step takes more time
Upvotes: 1
Related Questions
- How does loss.backward() relate to the appropriate parameters of the model?
- pytorch - connection between loss.backward() and optimizer.step()
- PyTorch — proper way to compute loss on GPU?
- Is loss.backward() meant to be called on each sample or on each batch?
- Difference between the calculation of the training loss and validation loss using pytorch
- loss function with pytorch
- Pytorch - Should backward() function be in the loop of epoch or batch?
- Which backpropagation step is the most expensive? (Tensorflow)
- pytorch: How does loss behave when coming from two networks?
- Is computing `loss.backward` for multiple losses performant in pytorch?