CODEWITHSUNDEEP
pythonmachine-learningkerasdeep-learning
Vishal Anand
Vishal Anand

Reputation: 178

How to make sure that Keras model weights are initialised randomly every-time the model is fit

I am training a Keras model for my data. I have to split the data into 3 parts and I am calling the same keras model for each split and trying to fit and predict consecutively.

I have a suspicion that every-time I call the model the model weights remain the same after reaching convergence from last training. And the next model called starts minimising the error from its previous state. I want that each time the model is trained, it starts to fit the data from a different random weights initialisation. Because all of my 3 splits are subset of the same dataset and I don't want any data leakage into the model due to seeing the split data beforehand while training.

Can I know if it is reinitialising the weights every-time the model is fit. And if not how can I do so?

here is how my code looks like



model = Sequential()
model.add(Dense(512, input_dim=77, kernel_initializer='RandomNormal', activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(256, kernel_initializer='RandomNormal', activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(512, kernel_initializer='RandomNormal', activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(256, kernel_initializer='RandomNormal', activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(512, kernel_initializer='RandomNormal', activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(256, kernel_initializer='RandomNormal', activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))


# Compile model
model.compile(loss='mean_absolute_error', optimizer='adam')


model()
# evaluate model
history = model.fit(scaler.transform(X_train_high), y_train_high,
                    batch_size=128,
                    epochs=5)
results = model.evaluate(scaler.transform(X_train_high), y_train_high, batch_size=128)
print('High test loss, test acc:', results)

# evaluate model
history = model.fit(scaler.transform(X_train_medium), y_train_medium,
                    batch_size=128,
                    epochs=5)
results = model.evaluate(scaler.transform(X_train_medium), y_train_medium, batch_size=128)
print(' Medium test loss, test acc:', results)

# evaluate model
history = model.fit(scaler.transform(X_train_low), y_train_low,
                    batch_size=128,
                    epochs=5)
results = model.evaluate(scaler.transform(X_train_low), y_train_low, batch_size=128, epochs=5)
print('Low test loss, test acc:', results)

Upvotes: 1

Views: 1478

Answers (1)

li.SQ
li.SQ

Reputation: 98

The model will keep its weight until you redefine one.

def define_model():
    model = Sequential()
    model.add(Dense(512, input_dim=77, kernel_initializer='RandomNormal', activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(256, kernel_initializer='RandomNormal', activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(512, kernel_initializer='RandomNormal', activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(256, kernel_initializer='RandomNormal', activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(512, kernel_initializer='RandomNormal', activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(256, kernel_initializer='RandomNormal', activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(1))

model=define_model()
# Compile model
model.compile(loss='mean_absolute_error', optimizer='adam')


# evaluate model
history = model.fit(scaler.transform(X_train_high), y_train_high,
                    batch_size=128,
                    epochs=5)
results = model.evaluate(scaler.transform(X_train_high), y_train_high, batch_size=128)
print('High test loss, test acc:', results)

model=define_model()

model.compile(loss='mean_absolute_error', optimizer='adam')
# evaluate model
history = model.fit(scaler.transform(X_train_medium), y_train_medium,
                    batch_size=128,
                    epochs=5)
results = model.evaluate(scaler.transform(X_train_medium), y_train_medium, batch_size=128)
print(' Medium test loss, test acc:', results)

You can check by model.get_weights.

Upvotes: 1

Related Questions