CODEWITHSUNDEEP
deep-learningcaffe
Yuval Atzmon
Yuval Atzmon

Reputation: 5945

How to change the learning rate of specific layer from the solver prototxt (CAFFE)

Anybody knows how to change the learning rate lr_mult of a specific layer in CAFFE from the solver prototxt? I know there's base_lr, however I would like to target the rate of a specific layer, and doing it from the solver instead of the network prototxt.

Thanks!

Upvotes: 1

Views: 1531

Answers (1)

mtngld
mtngld

Reputation: 577

Every layer that requiers learning (i.e convultional, fully-connected, etc.) has a specific lr_mult parameter that can be controlled specifically for that layer. lr_mult is a "multiplier on the global learning rate for this parameter."

Simply define or change the lr_mult for your layer in train_val.prototxt.

This is useful for fine-tuning, where you might want to have increased learning rate only for the new layer.

For more info check the caffe fine-tuning tutorial. (Note: it is a bit outdated and the deprecated term blobs_lr is used there instead of lr_mult)

EDIT: To my best knowledge it is not possible to define a layer-specific learning rate from the solver.prototxt. Hence, assuming the solver.prototxt limitation is not strict, I suggest a different method to achieve the same result.

Upvotes: 1

Related Questions