CODEWITHSUNDEEP
tensorflowkeraskeras-layertf.keras
christk
christk

Reputation: 872

Does freezing keras layers reduce memory needed for training?

I have pretrained weights of a large model that can't fit in my vram. If I freeze layers, the memory needed for training will be reduced? If so, the freezed layer weights aren't saved in vram?

Upvotes: 1

Views: 636

Answers (1)

Geeocode
Geeocode

Reputation: 5797

The frozen weights are parts of your model's parameters. So if you freeze them, the only thing will happen, that no calculation will be doing with them (note: this is tend to be true, but not always true), therefore the optimization will be faster. As they are still part of the whole optimizing process, they will be loaded in the GPU memory during the optimization(training) like they weren't frozen.

Upvotes: 2

Related Questions