Positional argument error when trying to train SGDClassifier on binary classification

Question

I'm working through Aurelien Geron's Hands-On ML textbook and have got stuck trying to train an SGDClassifier.

I'm using the MNIST handwritten numbers data and running my code in a Jupyter Notebook via Anaconda. Both my anaconda (1.7.0) and sklearn (0.20.dev0) are updated. I've pasted the code I used to load the data, select the first 60k rows, shuffle the order and convert the labels to 1 (True) for all 5's and 0 (False) for all other numbers. Both X_train and y_train_5 are numpy arrays.

I've pasted the error message I get below.

Nothing seems to be wrong with the dimensions of the data, I tried converting X_train to a sparse matrix (the suggested format for SGDClassifier) and various max_iter values and got the same error message each time. Am I missing something obvious? Do I need to use a different version of sklearn? I've searched online but couldn't find any posts describing similar issues with SGDClassifier. I'd be super grateful for any kind of pointer.

Code

from six.moves import urllib
from scipy.io import loadmat
import  numpy as np
from  sklearn.linear_model  import SGDClassifier


# Load MNIST data #

from scipy.io import loadmat
mnist_alternative_url = "https://github.com/amplab/datascience- 
sp14/raw/master/lab7/mldata/mnist-original.mat"
mnist_path = "./mnist-original.mat"
response = urllib.request.urlopen(mnist_alternative_url)
with open(mnist_path, "wb") as f:
    content = response.read()
    f.write(content)
mnist_raw = loadmat(mnist_path)
mnist = {
    "data": mnist_raw["data"].T,
    "target": mnist_raw["label"][0],
    "COL_NAMES": ["label", "data"],
    "DESCR": "mldata.org dataset: mnist-original",
}


# Assign X and y #

X, y = mnist['data'], mnist['target']


# Select first 60000 numbers #

X_train, X_test, y_train, y_test = X[:60000], X[60000:], y[:60000], 
y[60000:]


# Shuffle order #

shuffle_index  = np.random.permutation(60000)
X_train, y_train = X_train[shuffle_index], y_train[shuffle_index]


# Convert labels to binary (5 or "not 5") #

y_train_5 = (y_train == 5)
y_test_5 = (y_test == 5)

# Train SGDClassifier #

sgd_clf = SGDClassifier(max_iter=5, random_state=42)
sgd_clf.fit(X_train, y_train_5)

Error Message

---------------------------------------------------------------------------
TypeError
Traceback (most recent call last)
<ipython-input-10-5a25eed28833> in <module>()
     37 # Train SGDClassifier
     38 sgd_clf = SGDClassifier(max_iter=5, random_state=42)
---> 39 sgd_clf.fit(X_train, y_train_5)

~\Anaconda3\lib\site-packages\sklearn\linear_model\stochastic_gradient.py in fit(self, X, y, coef_init, intercept_init, sample_weight)
712                          loss=self.loss, learning_rate=self.learning_rate,
713                          coef_init=coef_init, intercept_init=intercept_init,
--> 714                          sample_weight=sample_weight)
    715 
    716 

~\Anaconda3\lib\site-packages\sklearn\linear_model\stochastic_gradient.py in _fit(self, X, y, alpha, C, loss, learning_rate, coef_init, intercept_init, sample_weight)
    570 
    571         self._partial_fit(X, y, alpha, C, loss, learning_rate, self._max_iter,
--> 572                           classes, sample_weight, coef_init, intercept_init)
    573 
    574         if (self._tol is not None and self._tol > -np.inf

~\Anaconda3\lib\site-packages\sklearn\linear_model\stochastic_gradient.py in _partial_fit(self, X, y, alpha, C, loss, learning_rate, max_iter, classes, sample_weight, coef_init, intercept_init)
    529                              learning_rate=learning_rate,
    530                              sample_weight=sample_weight,
--> 531                              max_iter=max_iter)
    532         else:
    533             raise ValueError(

~\Anaconda3\lib\site-packages\sklearn\linear_model\stochastic_gradient.py in _fit_binary(self, X, y, alpha, C, sample_weight, learning_rate, max_iter)
    587                                               self._expanded_class_weight[1],
    588                                               self._expanded_class_weight[0],
--> 589                                               sample_weight)
    590 
    591         self.t_ += n_iter_ * X.shape[0]

~\Anaconda3\lib\site-packages\sklearn\linear_model\stochastic_gradient.py in fit_binary(est, i, X, y, alpha, C, learning_rate, max_iter, pos_weight, neg_weight, sample_weight)
    419                            pos_weight, neg_weight,
    420                            learning_rate_type, est.eta0,
--> 421                            est.power_t, est.t_, intercept_decay)
    422 
    423     else:

~\Anaconda3\lib\site-packages\sklearn\linear_model\sgd_fast.pyx in sklearn.linear_model.sgd_fast.plain_sgd()

TypeError: plain_sgd() takes at most 21 positional arguments (25 given)

Answer 1

It appears your version of scikit-learn is just a little outdated. Try running:

pip install -U scikit-learn

then your code will run (with some slight formatting updates):

from six.moves import urllib
from scipy.io import loadmat
import numpy as np
from sklearn.linear_model  import SGDClassifier
from scipy.io import loadmat

# Load MNIST data #
mnist_alternative_url = "https://github.com/amplab/datascience-sp14/raw/master/lab7/mldata/mnist-original.mat"
mnist_path = "./mnist-original.mat"
response = urllib.request.urlopen(mnist_alternative_url)
with open(mnist_path, "wb") as f:
  content = response.read()
  f.write(content)
mnist_raw = loadmat(mnist_path)
mnist = {
  "data": mnist_raw["data"].T,
  "target": mnist_raw["label"][0],
  "COL_NAMES": ["label", "data"],
  "DESCR": "mldata.org dataset: mnist-original",
}

# Assign X and y #
X, y = mnist['data'], mnist['target']

# Select first 60000 numbers #
X_train, X_test, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]

# Shuffle order #
shuffle_index  = np.random.permutation(60000)
X_train, y_train = X_train[shuffle_index], y_train[shuffle_index]

# Convert labels to binary (5 or "not 5") #
y_train_5 = (y_train == 5)
y_test_5 = (y_test == 5)

# Train SGDClassifier #
sgd_clf = SGDClassifier(max_iter=5, random_state=42)
sgd_clf.fit(X_train, y_train_5)