CODEWITHSUNDEEP
scikit-learn
vkmv
vkmv

Reputation: 1395

Randomly hitting Joblib exception in Sklearn on parallel Grid Search

I am running gridSearchCV in parallel with n_jobs > 1, but randomly hit the following crash in joblib:

TypeError: Cannot create a consistent method resolution order (MRO) for bases JoblibException, Exception

Here is the complete stack trace:

      Traceback (most recent call last):
  File "example_sklearn.py", line 92, in <module>
    main()
  File "example_sklearn.py", line 76, in main
    ).fit(X_train, y_train)
  File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py", 
        line 372, in fit for clf_params in grid for train, test in cv)
  File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py",             
        line 516, in __call__self.retrieve()
  File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py", 
        line 448, in retrieve exception_type = _mk_exception(exception.etype)[0]
  File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/my_exceptions.py", 
        line 61, in _mk_exception__str__=JoblibException.__str__),
TypeError: Cannot create a consistent method resolution
order (MRO) for bases JoblibException, Exception

Any pointers on what this really is, and how I can debug this. Is this a known issue with sklearn

Upvotes: 1

Views: 1054

Answers (1)

adrin
adrin

Reputation: 4886

I had the exact same exception, exactly while using the GridSearchCV.

If you look at the exception, it is complaining about not being able to understand how exactly it should choose between two parent classes JoblibException and Exception. This is a bug in the joblib package, that the inheritance is improper.

But other than than, there exist another problem, which is the source of the exception itself. It's getting an exception while retrieve()ing, and while passing the exception, you get the error.

The second problem (which is the source of the exception), seems to be fixed in later versions of joblib. But scikit-learn is still using an old version (I will submit a pull request with the changed file soon).

A temporary workaround would be to install your own version of joblib using

easy_install joblib

and then go to the sklearn/exterlan folder, remove/rename the joblib folder, and create a symbolic link to your own joblib using:

ln -s /path/to/joblib joblib

EDIT: Seems somebody has had already fixed the problem. My version was also old.

Upvotes: 2

Related Questions