CODEWITHSUNDEEP
tensorflowkeraskeras-tuner
barhdi
barhdi

Reputation: 31

Keras Tuner Hyperband - how to set max trials and max epochs?

I have two questions regarding the Keras Tuner Hyperband class (for a regression problem)

tuner = kerastuner.tuners.hyperband.Hyperband(hypermodel,
                                  objective,
                                  max_epochs,
                                  factor=3,
                                  hyperband_iterations=1,
                                  seed=None,
                                  hyperparameters=None,
                                  tune_new_entries=True,
                                  allow_new_entries=True,
                                  **kwargs)

https://keras-team.github.io/keras-tuner/documentation/tuners/#hyperband-class

  1. Unlike BayesianOptimization and RandomSearch, the Tuner Hyperband (and Sklearn) does not have an argument 'max_trials'. What is the best way to define them? The documentation mentions a maximum of N*(log(N)/log(f))^2 cumulative epochs across all trials of (N=max_epochs, f=3 default) which seems very high considering that I usually need max_epochs > 10000 for a good training run. I would like to limit Keras Tuner computation time, for example to approximately one day. Is there a nicer way than canceling per ctrl+c in order to automatically start the training?

  2. When I start the tuning using

    tuner.search(
         x=trainX,
         y=trainY,
         validation_split=0.1,
         batch_size=batch_size,
         callbacks=[stop_early],
         epochs=max_epochs_search)

    another argument for the num of epochs can be passed. What is the relation between 'epochs' for 'search()' and 'max_epochs' for 'Hyperband()'? None of them or the formula N*(log(N)/log(f))^2 seems to fit the overall number of epochs.

I could not find much information elsewhere. Also after reading the paper this is not yet clear to me. Any hints are welcome. Thank you!

Upvotes: 3

Views: 5227

Answers (2)

Kaushik
Kaushik

Reputation: 1

This is from the official keras documentation: https://keras.io/keras_tuner/api/tuners/hyperband/

keras_tuner.Hyperband(
    hypermodel=None,
    objective=None,
    max_epochs=100,
    factor=3,
    hyperband_iterations=1,
    seed=None,
    hyperparameters=None,
    tune_new_entries=True,
    allow_new_entries=True,
    max_retries_per_trial=0,
    max_consecutive_failed_trials=3,
    **kwargs
)

even though max_trials cannot be explicitly passed as an argument in hyperband, max_epochs, factor, and hyperband_iterations influence the number of trials the tuner runs. try tuning those 3 arguments.

Upvotes: 0

SirIcarus
SirIcarus

Reputation: 371

Regarding your second question:

According to this, it still seems to be an open issue and looks like the 'epochs' argument of the search()-method is redundant.

Upvotes: 1

Related Questions