CODEWITHSUNDEEP
pythontensorflowkeras
jdbrody
jdbrody

Reputation: 513

Greedy Layerwise Training with Keras

I'm trying to implement a multi-layer perceptron in Keras (version 2.2.4-tf) that trains layers iteratively. I've been using https://machinelearningmastery.com/greedy-layer-wise-pretraining-tutorial/ as a model, but things aren't quite working for me. Here's my code:

# Train first layer
mlp = keras.models.Sequential()
mlp.add(keras.layers.Dense(units=512, activation='tanh', kernel_initializer='he_uniform'))
mlp.add(keras.layers.Dense(units=1, activation='sigmoid', kernel_initializer='he_uniform'))
mlp.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
mlp.fit(X,y)
output_layer = mlp.layers[-1]

# Add and train second layer
mlp.pop()
for layer in mlp.layers:
    layer.trainable=False
mlp.add(keras.layers.Dense(units=512, activation='tanh', kernel_initializer='he_uniform'))
mlp.add(output_layer)
mlp.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
mlp.fit(X,y)

The error I get is 

 ('Error when checking model target: expected no data, but got:', array([1., 0., 0., 1., 1., 0., 0., 0., 1., 0., 1., 0., 1., 0., 1., 1., 1.,
       0., 0., 1., 1., 0., 0., 1., 0., 0., 0., 1., 1., 1., 1., 0., 0., 1.,
       1., 0., 0., 1., 1., 1., 0., 0., 1., 1., 1., 1., 0., 0., 1., 1., 1.,
       0., 0., 1., 1., 1., 1., 0., 0., 1., 0., 0., 0., 1., 1., 1., 1., 1.,
       0., 0., 0., 1., 0.], dtype=float32))

If I try commenting out the second compile I instead get

ValueError: Weights for model sequential_3 have not yet been created. Weights are created when the Model is first called on inputs or `build()` is called with an `input_shape`.

Does anyone know what I'm missing here?

In case it's relevant, I'm running Keras with a Tensorflow-1.14.0 backend, installed via conda from mkl repo. 

~ $ conda list tensorflow
# packages in environment at /home/justin/.conda/envs/local_idp:
#
tensorflow                1.14.0          mkl_py36h2526735_0  
tensorflow-base           1.14.0          mkl_py36h7ce6ba3_0  
tensorflow-estimator      1.14.0                     py_0

Many thanks,

Justin

Upvotes: 1

Views: 592

Answers (1)

Manoj Mohan
Manoj Mohan

Reputation: 6044

I was able to reproduce the issue. The issue is happening because of using keras from the tensorflow namespace. Replace 

import tensorflow.keras as keras

with

import keras

Upvotes: 1

Related Questions