CODEWITHSUNDEEP
pythonregressionscikit-learncross-validation
ahmethungari
ahmethungari

Reputation: 2229

scikit-learn cross validation, negative values with mean squared error

When I use the following code with Data matrix X of size (952,144) and output vector y of size (952), mean_squared_error metric returns negative values, which is unexpected. Do you have any idea?

from sklearn.svm import SVR
from sklearn import cross_validation as CV

reg = SVR(C=1., epsilon=0.1, kernel='rbf')
scores = CV.cross_val_score(reg, X, y, cv=10, scoring='mean_squared_error')

all values in scores are then negative.

Upvotes: 71

Views: 110419

Answers (3)

MD Rijwan
MD Rijwan

Reputation: 481

To see what are available scoring keys use:

import sklearn
print(sklearn.metrics.SCORERS.keys())

You can either use 'r2' or 'neg_mean_squared_error'. There are lots of options based on your requirement.

Upvotes: 1

Otacílio Maia
Otacílio Maia

Reputation: 309

You can fix it by changing scoring method to "neg_mean_squared_error" as you can see below:

from sklearn.svm import SVR
from sklearn import cross_validation as CV

reg = SVR(C=1., epsilon=0.1, kernel='rbf')
scores = CV.cross_val_score(reg, X, y, cv=10, scoring='neg_mean_squared_error')

Upvotes: 3

AN6U5
AN6U5

Reputation: 2885

Trying to close this out, so am providing the answer that David and larsmans have eloquently described in the comments section:

Yes, this is supposed to happen. The actual MSE is simply the positive version of the number you're getting.

The unified scoring API always maximizes the score, so scores which need to be minimized are negated in order for the unified scoring API to work correctly. The score that is returned is therefore negated when it is a score that should be minimized and left positive if it is a score that should be maximized.

This is also described in sklearn GridSearchCV with Pipeline.

Upvotes: 96

Related Questions