CODEWITHSUNDEEP
tensorflowkerasdeep-learning
Hyunkyu Park
Hyunkyu Park

Reputation: 33

How to solve the problem that Trainable-False does not work in Keras?

I am now facing malfunction of "trainable=false".

When I developed a code with a structure that,

the model has two subdivided model(FC model, CN model) and those are connected in a serial way.

After training FC model only, I want to freeze the FC and train FC+CN, the entire model.

However the freezing with trainable does not work, and something strange is coming out.

When not freeze :

model.FCnetwork.trainable = True
model.FCnetwork.summary()

Total params: 2,584,576
Trainable params: 2,578,432
Non-trainable params: 6,144

and when freeze :

model.FCnetwork.trainable = False
model.FCnetwork.summary()

Total params: 5,163,008
Trainable params: 2,578,432
Non-trainable params: 2,584,576

Total params are increased. And of course, the freezing does not work.

This is my designed class

class MYMAP():
    def __init__(self):
        # Input shape


        optimizer = optimizers.Adam()

        self.CNnetwork= self.Convolutional_network()
        self.CNnetwork.compile()




        self.FCnetwork = self.Fullyconnected_network()
        self.FCnetwork.compile(loss='mse',
            optimizer=optimizer)

        z = Input(shape=(input_size,))
        img = self.FCnetwork(z)

        valid = self.CNnetwork(img)

        self.combined = Model(z, valid)

        optimizer_DG = optimizers.Adam()
        self.combined.compile(loss='mse', optimizer=optimizer_DG)

    def Fullyconnected_network(self):

        noise = Input(shape=(input_size,))
        img = model(noise)

        return Model(noise, img)




    def Convolutional_network(self):

        img = Input(shape=(image_size_vectored,))
        validity = model(img)

        return Model(img, validity)

It was bit hard for me to figure out the way to solve.

Thank you so much.

Upvotes: 2

Views: 1429

Answers (1)

mujjiga
mujjiga

Reputation: 16906

As the warning says it clearly

did you set model.trainable without calling model.compile

Correct sample code:

class MYMAP():
    def __init__(self):        
        self.optimizer = optimizers.Adam()
        self.FCnetwork = self.Fullyconnected_network()

        self.FCnetwork.compile(loss='mse',
            optimizer=self.optimizer)

        z = Input(shape=(32,))
        img = self.FCnetwork(z)


    def Fullyconnected_network(self):            
        noise = Input(shape=(32,))        
        img = Dense(8)(noise)
        return Model(noise, img)

model = MYMAP()
model.FCnetwork.trainable = True
model.FCnetwork.compile(loss='mse', optimizer=optimizers.Adam())
model.FCnetwork.summary()
model.FCnetwork.trainable = False
model.FCnetwork.compile(loss='mse', optimizer=optimizers.Adam())
model.FCnetwork.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_39 (InputLayer)        (None, 32)                0         
_________________________________________________________________
dense_15 (Dense)             (None, 8)                 264       
=================================================================
Total params: 264
Trainable params: 264
Non-trainable params: 0
_________________________________________________________________
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_39 (InputLayer)        (None, 32)                0         
_________________________________________________________________
dense_15 (Dense)             (None, 8)                 264       
=================================================================
Total params: 264
Trainable params: 0

So make sure to run model.compile after you change the trainable parameter of the model.

Upvotes: 2

Related Questions