CODEWITHSUNDEEP
pythonmachine-learningscikit-learn
Marvin Krüger
Marvin Krüger

Reputation: 137

Sklearn RandomizedSearchCV OSError: [Errno 5] Input/output error

I'm trying to use a RandomizedSearch to determine the best hypterparameters for SKLearn's MLP and XGBoost. While running the optimization, after roughly 50 runs an OSError occurred.

The code I used for randomizedSearch with XGBoost:

from scipy.stats import randint as sp_randint   
import xgboost as xgb
from sklearn.model_selection import RandomizedSearchCV
from joblib import dump, load
from sklearn.model_selection import StratifiedShuffleSplit
import numpy as np
import pickle
# Reference f1 eval --> https://stackoverflow.com/questions/51587535/custom-evaluation-function-based-on-f1-for-use-in-xgboost-python-api
from sklearn.metrics import f1_score
import numpy as np

def f1_eval(y_pred, dtrain):
    y_true = dtrain.get_label()
    err = 1-f1_score(y_true, np.round(y_pred))
    print("Score: ", str(1 - err))
    return 'f1_err', err

neg_samples = len(y[y['canceled_in_6_mon'] == 0])
pos_samples = len(y[y['canceled_in_6_mon'] == 1])

xgb_model = xgb.XGBClassifier(objective= 'reg:logistic', nthread=1,
scale_pos_weight=neg_samples / pos_samples)

parameters = {
    'learning_rate': [0.005, 0.01, 0.05, 0.1, 0.15, 0.25, 0.35], #so called `eta` value
    'max_depth': sp_randint(10, 100),
    'min_child_weight': sp_randint(1, 8),
    'silent': [1],
    'gamma': [0, 0.2, 0.5, 0.7, 1],
    'subsample': [0.5, 0.7, 1],
    'colsample_bytree': [0.5, 0.7, 1],
    "n_estimators": sp_randint(20, 100), 
    "max_features": sp_randint(10, 400),
    "min_samples_split": sp_randint(2, 20),
    "seed": [42],
    "min_samples_leaf": sp_randint(1, 5)
}

ss = StratifiedShuffleSplit(n_splits=3, test_size=0.24, random_state=42)

clf = RandomizedSearchCV(estimator=xgb_model,  param_distributions = parameters,cv=ss, verbose=10, n_jobs=4, scoring='f1', n_iter=50)


clf.fit(X=X_train, y=np.ravel(y_train), eval_metric=f1_eval)

Here's the complete output I got including the error: Pastebin Link

Any ideas why this happens? The error indicates that it's a problem with joblib, but I executed the same randomizedSearch code with a RandomForest Classifier and all works fine.

Upvotes: 2

Views: 512

Answers (1)

Mgaraz
Mgaraz

Reputation: 81

I came across the same problem and I found that replacing the existing xgboost.dll in the xgboost folder with one of the binaries from http://www.picnet.com.au/blogs/guido/2016/09/22/xgboost-windows-x64-binaries-for-download/ solved my issue.

Upvotes: 1

Related Questions