CODEWITHSUNDEEP
optimizationmachine-learninglogistic-regressiongradient-descenthyperparameters
giando.c
giando.c

Reputation: 66

Predictive Maintenance - How to use Bayesian Optimization with objective function and Logistic Regression with Gradient Descent together?

I'm trying to reproduce the problem shown in arimo.com

This is an example how to build a preventive maintenance Machine Learning model for an Hard Drive failures. The section I really don't understand is how to use Bayesian Optimization with a custom objective function and Logistic Regression with Gradient Descent together. What are the hyper-parameters to be optimized? What is the flow of the problem?

As described in our previous post, Bayesian Optimization [6] is used to find the best hyperparameter values. The objective function to be optimized in the hyperparameter tuning is the following score measured on the validation set:

S = alpha * fnr + (1 – alpha) * fpr

where fpr and fnr are the False Positive and False Negative rates obtained on the validation set. Our goal is to keep False Positive rate low, therefore we use alpha = 0.2. Since the validation set is highly unbalanced, we found out that standard scores like Precision, F1-score, etc… do not work well. In fact, using this custom score is crucial for the model to obtain a good performance generally.

Note that we only use the above score when running Bayesian Optimization. To train logistic regression models, we use Gradient Descent with the usual ridge loss function.

My dataframe before features selection:

index   date    serial_number   model   capacity_bytes  failure     Read Error Rate     Reallocated Sectors Count   Power-On Hours (POH)    Temperature     Current Pending Sector Count    age     yesterday_temperature   yesterday_age   yesterday_reallocated_sectors_count     yesterday_read_error_rate   yesterday_current_pending_sector_count  yesterday_power_on_hours    tomorrow_failure
0   77947   2013-04-11  MJ0331YNG69A0A  Hitachi HDS5C3030ALA630     3000592982016   0   0   0   4909    29  0   36348284.0  29.0    20799895.0  0.0     0.0     0.0     4885.0  0.0
1   79327   2013-04-11  MJ1311YNG7EWXA  Hitachi HDS5C3030ALA630     3000592982016   0   0   0   8831    24  0   36829839.0  24.0    21280074.0  0.0     0.0     0.0     8807.0  0.0
2   79592   2013-04-11  MJ1311YNG2ZD9A  Hitachi HDS5C3030ALA630     3000592982016   0   0   0   13732   26  0   36924206.0  26.0    21374176.0  0.0     0.0     0.0     13708.0     0.0
3   80715   2013-04-11  MJ1311YNG2ZDBA  Hitachi HDS5C3030ALA630     3000592982016   0   0   0   12745   27  0   37313742.0  27.0    21762591.0  0.0     0.0     0.0     12721.0     0.0
4   79958   2013-04-11  MJ1323YNG1EK0C  Hitachi HDS5C3030ALA630     3000592982016   0   524289  0   13922   27  0   37050016.0  27.0    21499620.0  0.0     0.0     0.0     13898.0     0.0

Upvotes: 2

Views: 413

Answers (1)

K3---rnc
K3---rnc

Reputation: 7049

As is the case in code example of SAMBO Optimizer, the Bayesian optimization process can use any underlying surrogate model (estimator) that models the unknown, true space.

SAMBO uses Gaussian process regression estimator from scikit-learn by default, but you could also specify, as per your source citation, estimator=LogisticRegression() or similar.

Upvotes: 0

Related Questions