CODEWITHSUNDEEP
pythontensorflowmachine-learningkeras
leiferz
leiferz

Reputation: 27

Keras: Reusing weights for several layers

I am trying to implement a neuronal network with the Keras functional API, which uses the same weights for several layers. The code is working, but I am not sure whether the "shared layers" which I created do what I want. Do the two hidden layers in the example use the same weights or have I created two different instances of one layer, which have only the structure in common? If not, is there a way to achieve what I want?

# create shared_layer
inputs = Input(shape=(784,))
outputs = layers.Dense(784, activation='relu')(inputs)
shared_layer = Model(inputs=inputs, outputs=outputs)

# create model
visible = Input(shape=(28, 28, 1))
flat = layers.Flatten()(visible)
hidden = shared_layer(flat)
hidden2 = shared_layer(hidden)
output = layers.Dense(10, activation='softmax')(hidden2)

new_model = Model(inputs=visible, outputs=output)

When I look at the summary of the model I am getting this:

Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_4 (InputLayer)            (None, 28, 28, 1)    0                                            
__________________________________________________________________________________________________
flatten_2 (Flatten)             (None, 784)          0           input_4[0][0]                    
__________________________________________________________________________________________________
model_3 (Model)                 (None, 784)          615440      flatten_2[0][0]                  
                                                                 model_3[1][0]                    
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 10)           7850        model_3[2][0]                    
==================================================================================================

Upvotes: 0

Views: 393

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86650

It's shared, but you're doing unnecessary things.

You can:

shared_layer = layers.Dense(784, activation='relu')

visible = Input(shape=(28, 28, 1))
flat = layers.Flatten()(visible)
hidden = shared_layer(flat)
hidden2 = shared_layer(hidden)
output = layers.Dense(10, activation='softmax')(hidden2)

new_model = Model(inputs=visible, outputs=output)

Upvotes: 1

Related Questions