CODEWITHSUNDEEP
pythontensorflowkerasoptuna
Talha Anwar
Talha Anwar

Reputation: 2949

Unable to produce optuna results keras

I have created a model and tuned with optuna

def mymodel(hp):
    clear_session()
    imageModel = Sequential() 
    imageModel.add(Conv2D(hp.suggest_categorical("kernel1", [32,64,128]),
                          hp.suggest_categorical("filter1", [3,5]),
                          activation='relu', padding='valid', strides=2,input_shape=(300, 300, 1)))
    imageModel.add(MaxPool2D(pool_size=2, strides=2))
   
    imageModel.add(Flatten())
    imageModel.add(Dense(hp.suggest_categorical("dense", [32,64,128]), activation='relu'))
  
    imageModel.add(Dense(1, activation='sigmoid'))  

    imageModel.compile(optimizer=RMSprop(),loss='binary_crossentropy',metrics=['accuracy'])
    imageModel.fit(XTrain,YTrain,epochs = 20,validation_data=(XVal,YVal),verbose=0)
    result=imageModel.evaluate(XTest,YTest)
    return result[1]

Tuned it this way

if __name__ == "__main__":
    
    study = optuna.create_study(direction="maximize")
    study.optimize(mymodel, n_trials=100, timeout=6000)

    print("Number of finished trials: {}".format(len(study.trials)))

    print("Best trial:")
    trial = study.best_trial

    print("  Value: {}".format(trial.value))

    print("  Params: ")
    for key, value in trial.params.items():
        print("    {}: {}".format(key, value))

Now let say it tuned and give me some parameters and i put them in architecture and train model again using keras fit function, it would not reproduce the results.

def mymodel():
    clear_session()
    imageModel = Sequential() 
    imageModel.add(Conv2D(32,3,activation='relu', padding='valid', strides=2,input_shape=(300, 300, 1)))
    imageModel.add(MaxPool2D(pool_size=2, strides=2))
   
    imageModel.add(Flatten())
    imageModel.add(Dense(64, activation='relu'))
  
    imageModel.add(Dense(1, activation='sigmoid'))  

    imageModel.compile(optimizer=RMSprop(),loss='binary_crossentropy',metrics=['accuracy'])
    imageModel.fit(XTrain,YTrain,epochs = 20,validation_data=(XVal,YVal),verbose=0)
    result=imageModel.evaluate(XTest,YTest)
    return result[1]
mymodel()

In the start of notebook i have put following lines

import os
import numpy as np
import tensorflow as tf
import random
seed=0
os.environ['PYTHONHASHSEED'] = str(seed)
np.random.seed(seed)
random.seed(seed) 
tf.random.set_seed(seed)

How can i handle this issue

Upvotes: 0

Views: 1906

Answers (1)

nzw0301
nzw0301

Reputation: 399

To fix optuna's result, we need to replace

study = optuna.create_study(direction="maximize")

with

study = optuna.create_study(direction="maximize", sampler=optuna.samplers.TPESampler(seed=seed))

See https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-obtain-reproducible-optimization-results

Upvotes: 3

Related Questions