CODEWITHSUNDEEP
pythonmachine-learningcomputer-visiontensorflowdeep-learning
Wassim Gr
Wassim Gr

Reputation: 588

Fine-tuning a deep neural network in Tensorflow

I want to partially fine-tune a pre-trained deep neural network in Tensorflow (as in, load weights for all layers but only update weights on higher level layers).

Is there any method in Tensorflow that allows the selection of variables that should be changed and those that should be kept the same?

Thank you in advance!

Upvotes: 3

Views: 2892

Answers (1)

mrry
mrry

Reputation: 126194

When you create an optimizer (e.g. tf.train.AdagradOptimizer) to train your model, you can pass an explicit var_list=[...] argument to the Optimizer.minimize() method. (If you don't specify this list, it will default to containing all of the variables in tf.trainable_variables().)

For example, depending on your model, you might be able to use the names of your variables to define the list of variables to be optimized:

# Assuming all variables to be fine-tuned have a name that starts with
# "layer17/".
opt_vars = [v for v in tf.trainable_variables() if v.name.startswith("layer17/")]

train_op = optimizer.minimize(loss, var_list=opt_vars)

Upvotes: 7

Related Questions