CODEWITHSUNDEEP
pythonscikit-learnkeras
WolVes
WolVes

Reputation: 1336

Keras - How to actually use an autoencoder

I have developed the following autoencoder in Keras for the purpose of dimension reduction. 

from keras.layers import Input, Dense, BatchNormalization
from keras.models import Model
from keras import regularizers
from keras.callbacks import TensorBoard, EarlyStopping
import keras.backend as K
from sklearn.metrics import r2_score

input_size = len(spot_dat.columns)
coder_size = 32
inner_size = 64
betwe_size = 96
outer_size = 128
batch_size = 25


def r2(y_true, y_pred):
    SS_res = K.sum(K.square(y_true - y_pred))
    SS_tot = K.sum(K.square(y_true - K.mean(y_true)))
    return (1 - SS_res / (SS_tot + K.epsilon()))

def rmse(y_true, y_pred):
    return K.sqrt(K.mean(K.square(y_pred - y_true), axis=-1))

input_ = Input(shape=(input_size,))
encoded = Dense(outer_size, activation='selu')(input_) #hidden 1
#encoded = Dense(betwe_size, activation='elu')(input_) #hidden 2
encoded = BatchNormalization()(encoded)
encoded = Dense(inner_size, activation='selu')(encoded) #hidden 3

code = Dense(coder_size, activation='selu')(encoded) #code 


decoded = Dense(inner_size, activation='selu')(code) #hidden 2
decoded = BatchNormalization()(decoded)
#decoded = Dense(betwe_size, activation='elu')(decoded) #hidden 2
decoded = Dense(outer_size, activation='selu')(decoded) #hidden 1
output = Dense(input_size, activation='sigmoid')(decoded) #output

autoencoder = Model(input_, output)
autoencoder.compile(optimizer = 'adam', loss = 'mean_squared_error', metrics = [r2, rmse])
val = autoencoder.fit(x_train, x_train, 
                epochs=1000,
                batch_size = 75,
                shuffle=True,
                validation_data=(x_test, x_test),
                callbacks=[TensorBoard(log_dir='/tmp/test'), EarlyStopping(monitor = 'val_loss', min_delta = 0, patience = 30, verbose = True, mode = 'auto')])

plt.plot(val.history['loss'])
plt.plot(val.history['val_loss'])
plt.title('Model loss (MSR)')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc = 'best')
plt.show()

plt.plot(val.history['r2'])
plt.plot(val.history['val_r2'])
plt.title('Model R2')
plt.ylabel('R2')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc = 'best')
plt.show()

plt.plot(val.history['rmse'])
plt.plot(val.history['val_rmse'])
plt.title('Model RMSE')
plt.ylabel('RMSE')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc = 'best')
plt.show()

The model works fine, the issue is I cannot find anyway to extract part of the model (from input to code) after it has been trained on the full autoencoder. It seems to me, if you cant pull half the model the autoencoder cannot be used for any practical dimension reduction. Thus, there should be a way to freeze and pull the trained models first several layers. The end goal is to apply the layer with 32 dimensions to a t-SNE and compare results against other dimension reduction techniques.

Upvotes: 0

Views: 525

Answers (1)

Dr. Snoopy
Dr. Snoopy

Reputation: 56357

After training just do the following:

encoder = Model(input_, code)

Then use encoder.predict to get the code from an input sample.

Upvotes: 2

Related Questions