CODEWITHSUNDEEP
pythonscikit-learnkeras
ThisIsMyDisplayName_
ThisIsMyDisplayName_

Reputation: 31

How to scale new data coming into a neural network after it's been trained

I have a trained neural network that works really well, but I scaled the data beforehand [sklearn preprocessing.scale(X)]. This works great, but what do I do when I save the model and want to input new data into it? How do I make sure the scaling is the same as when it was trained?

df = pd.read_csv("Trimmed Training Data.csv", delimiter=",")

X = np.array(df.drop(['PredictThis'],1))
y = np.array(df['PredictThis'])

X = preprocessing.scale(X)


# create NN model
model = Sequential()
# 2 inputs, 10 neurons in 1 hidden layer, with tanh activation and dropout
model.add(Dropout(0.2, input_shape=(15,)))
model.add(Dense(100, init='uniform', input_shape=(15,), activation='relu')) 
model.add(Dropout(0.2))
model.add(Dense(100, init='uniform', input_shape=(100,), activation='relu')) 
model.add(Dropout(0.2))
model.add(Dense(5, init='uniform', input_shape=(100,), activation='relu')) 
model.add(Dropout(0.2))
model.add(Dense(1, init='uniform', activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop')

# Fit the model
model.fit(X, y, nb_epoch=25, batch_size=15)

Upvotes: 3

Views: 476

Answers (1)

Marcin Możejko
Marcin Możejko

Reputation: 40516

What is usually done is saving a scale object as you may read here.

Here is how you could save scale object:

from sklearn.externals import joblib
joblib.dump(scale, 'filename.pkl')

and here is how you could reload it:

scale = joblib.load('filename.pkl')

When predicting new data - it's the best to reload scaler - scale data accordingly and then to use your model. You could even create a new class with scaler kept and saved like keras.model.

Upvotes: 2

Related Questions