CODEWITHSUNDEEP
kerasconv-neural-network
dumb_coder
dumb_coder

Reputation: 325

Weights transfer from one model to another model with different layer architecture

I have a CNN network with three layers and after fitting the model, weights are saved on disk. The second time I load the weights, but this time model is increased with a layer. So it is now 4 layer network. Is it possible to transfer model weights with the different architecture? If yes then how? I am using keras for development.

For me it shows error: 'You are trying to load a weight file containing 3 layers into a model with 4 layers'.

Thanks in advance!

Upvotes: 5

Views: 7560

Answers (2)

Susmit Agrawal
Susmit Agrawal

Reputation: 3764

I'm a bit late at answering, but hopefully this will help somebody.

This is how I did it:

  1. Use a list to store all layers:

    model_layers = []
model_layers.append(keras.layers.Conv2D(...))
...
model_layers.append(keras.layers.Dense(units=num_classes))
model_layers.append(keras.layers.Softmax())

  2. Define the source model, and add layers from the list. Load weights from the saved file:

    model = keras.Sequential()
for layer in model_layers:
    model.add(layer)
model.compile(...)
model.load_weights(filename)

  3. Copy the original list to a new temporary list. Clear the original list, and to it add new instances of layers required for the target network:

    temp_layers = model_layers.copy()
model_layers.clear()
model_layers.append(keras.layers.Conv2D(...))
...
model_layers.append(keras.layers.Dense(units=num_classes))
model_layers.append(keras.layers.Softmax())

  4. Assuming all layers in the source network form the initial part of the target network, copy all layer references form the temporary list to model_layers. EXCLUDE THE CLASSIFICATION LAYERS if required. You can also restore individual layers if you know their indices:

    for i in range(len(temp_layers) - 2):
    model_layers[i] = temp_layers[i]

  5. Create the new model by following step 2:

    new_model = keras.Sequential()
for layer in model_layers:
    new_model.add(layer)

  6. Add any additional layers that may be required (along with the classification layer, if required):

    new_model.add(...)
new_model.add(keras.layers.Dense(units=num_classes))
new_model.add(keras.layers.Softmax())
new_model.compile(...)

I realise this is not a general answer. I have mentioned the exact steps I used in my successful implementation.

Feel free to try out different variations.

Upvotes: -1

Manto
Manto

Reputation: 31

I have not tried this, but it should be possible by using the layer.get_weights() and layer.set_weights(weights) methods.

weights = old_model_layer.get_weights()
new_model_layer.set_weights(weights)

See https://keras.io/layers/about-keras-layers/ for more info.

Upvotes: 3

Related Questions