CODEWITHSUNDEEP
machine-learningoptimizationneural-networkartificial-intelligencehyperparameters
Yongjun Lee
Yongjun Lee

Reputation: 1025

Choosing hyperparameters automatically

I'm studying neural networks, and regularization. I noticed that there are a lot of hyperparameters that needs to be set manually depending on the type of problem you're trying to solve, data sets that you're dealing with, etc.

I'm wondering if there is a way we can create another optimization model so that it automatically finds the best hyperparameters needed for training? I mean, we are able to find the appropriate weights by gradient descent, or other methods. So, if we are able to find some kind of cost function on the hyperparameters, won't we be able to do the same? Basically like an AI program adjusting our AI model.

Is something like this possible, or at least a valid question to ask? Maybe is this already done in some research that I'm not aware of?

Upvotes: 1

Views: 516

Answers (2)

siddharth ranjan
siddharth ranjan

Reputation: 113

There are some packages you can explore. I would recommend you to use talos. Please find the link below:

https://www.analyticsvidhya.com/blog/2021/05/neural-network-and-hyperparameter-optimization-using-talos/

At the last you will find the codes for talos

Upvotes: 0

Crissman Loomis
Crissman Loomis

Reputation: 156

Yes, there are frameworks available that will automatically tune hyper parameters for you. I'm on the Optuna team, and Optuna (optuna.org) is one such framework that will do Bayesian optimization to adjust hyper parameters for you. It can work with any python program that can be written with a loss or accuracy output.

Upvotes: 1

Related Questions