JobHunter69
Reputation: 2290
Why does not using retain_graph=True result in error?
If I need to backpropagate through a neural network twice and I don't use retain_graph=True, I get an error.
Why? I realize it is nice to keep the intermediate variables used for the first backpropagation to be reused for the second backpropagation. However, why aren't they simply recalculated, like they were originally calculated in the first backpropagation?
Upvotes: 0
Views: 91
Answers (1)
trsvchn
Reputation: 8991
By default, PyTorch doesn't store intermediate gradients, because the PyTorch's main feature is Dynamic Computational Graphs, so after backpropagation the graph will be freed all the intermediate buffers will be destroyed.
Upvotes: 0
Related Questions
- PyTorch - RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed
- RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed
- PyTorch error in trying to backward through the graph a second time
- retain_graph problem with GRU in Pytorch 1.6
- How does PyTorch's loss.backward() work when "retain_graph=True" is specified?
- RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True
- PyTorch: Simple feedforward neural network not running without retain_graph=True
- Why does setting backward(retain_graph=True) use up lot GPU memory?
- pyTorch can backward twice without setting retain_graph=True
- Pytorch - why does preallocating memory cause "trying to backward through the graph a second time"