Perceptron online training (scikit-learn)

Question

i wrote a simple program to classify a set of linearly separable 2D random points. I used a perceptron and i trained it with the fit method. Now i'd like to train the perceptron one point at time, plotting each time the hyperplane (a line in this case) using the updated weights. What i want to obtain is an animation which shows how the line become more and more precise dividing the sets. The fit method takes the entire training set, what about the partial_fit? Can i make a loop where i feed the method each time with a new single couple of input/output, and read continuously the coef_ and intercept_?

I read the documentation here http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html but i have some doubts on how to implement it.

EDIT 1

Thanks to Vivek Kumar, i implemented the partial_fit method to my code. The program creates 2 sets of coordinates, and for each couple produces an output which is 1 if the point is over a line and -1 if it is under the line. The code works with the fit method, but this versions gives some problems with the data shape. I tried to use reashape for the X data without any improvement.

import numpy as np
import matplotlib.pyplot as plt

def createLinearSet(nCamp, mTest, qTest):

    y_ = []
    X_ = np.random.rand(nCamp, 2)*20-10

    for n in range(nCamp):

        if X_[n][1] >= mTest*X_[n][0]+qTest :
            y_.append(1)
        else:
            y_.append(-1)

    return X_, y_

########################################################################
# VARIABLES
iterazioni = 100
eta = 0.6
y = []
error = []

########################################################################
# CREATING DATA SET
m_test = -2
q_test = 3
n_camp = 100

X, y = createLinearSet(n_camp, m_test, q_test)

########################################################################
# 70 % training data and 30 % test data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3,           random_state = 0)

########################################################################
# Data normalization
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
sc.fit(X_train)                       # Calcola la media dei campioni e la deviazione standard
X_train_std = sc.transform(X_train)   # Normalizza i dati di test e di addestramento
X_test_std = sc.transform(X_test)     # NB. uso media e deviazione dei dati di add. per entrambi,  
                                      #     così sono confrontabili
########################################################################
# Perceptron initialization
from sklearn.linear_model import Perceptron
ppn = Perceptron(n_iter = iterazioni, eta0 = eta, random_state = 0)

########################################################################
# Online training
num_samples = X_train_std.shape[0]
classes_y =  np.unique(y_train)

X_train_std = X_train_std.reshape(-1, 2)

for i in range(num_samples):
    ppn.partial_fit(X_train_std[i], y_train[i], classes = classes_y )

########################################################################
# Using test data for evaluation
y_pred = ppn.predict(X_test_std)

########################################################################
# Previsions accuracy
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test, y_pred) * 100

print("Accuracy: {} %".format(round(accuracy,2)))

print(ppn.coef_, ppn.intercept_) 

As you can see, the problem is in the "Online training" section. The error is:

/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

From documentation, X must be: X : {array-like, sparse matrix}, shape (n_samples, n_features)

If i print a single sample of X, the output is: [-0.25547959 -1.4763508 ]

Where is the error?

EDIT 2

Putting the line X_train_std[i].reshape(1,-1) in the loop it gives me this message: Traceback (most recent call last): File "Perceptron_Retta_Online.py", line 57, in <module> ppn.partial_fit(X_train_std[i].reshape(1,-1), y_train[i], classes = classes_y ) File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 512, in partial_fit coef_init=None, intercept_init=None) File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 344, in _partial_fit X, y = check_X_y(X, y, 'csr', dtype=np.float64, order="C") File "/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py", line 526, in check_X_y y = column_or_1d(y, warn=True) File "/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py", line 562, in column_or_1d raise ValueError("bad input shape {0}".format(shape)) ValueError: bad input shape ()

Answer 1

Most scikit estimators work on the full data at once, ie. when you call the fit() method, the older training (weights, coefficients etc) are lost, and model is fit on the new data only.

So to implement it like you said, you are correct that "i make a loop where i feed the method each time with a new single couple of input/output, and read continuously the coef_ and intercept_", but with a slight change. Instead of feeding the new single couple of input/ output, you need to feed all previous values and the new single values.

To illustrate my point, what you can do is following (This is somewhat pseudo coded, so read along the comments):

X = your_input
y = your_output
num_samples = X.shape[0]
for i in range(num_samples):

    # 1) This is necessary to initialize a new estimator everytime
    numpy.random.seed

    # 2) This is necessary to initialize a new estimator everytime
    estimator = MLPClassifier() OR SGDClassifier() OR ...

    # 3) Call fit on data from 0 to current index i
    estimator.fit(X[:i+1], y[:i+1])

    # Read the following values and do what you want
    estimator.coeff_ OR estimator.intercept_

Explanation: 1) Setting the random seed parameter inside the loop, so the initialization of weights in same (to same values) in all loops. See point 2.

2) Initialization of new object each time, instead of using same object (initialized out of the loop), so that the initial weights assigned to it are random (and fixed due to point 1).

3) Use all the values from 0 to current index to learning is as you want. For first loop its only 1st values (index 0), for 2nd loop its 1st and 2nd values, and so on ...

This approach will work for all scikit-estimators but takes a lot of time (if the data is very large) because of repeated training of same data points.

ALTERNATIVE: As you linked the SGDClassifier, so I assume you may be interested in Online-learning or Out-of-core learning, which serves your requirement exactly. But not all estimators are able to do this. Only estimators which are listed in the official documentation are capable of doing this.

They implement a partial_fit() method, which is what you want. So for them the code will be like this:

X = your_input
y = your_output
num_samples = X.shape[0]

estimator = SGDClassifier() OR ... (which implements partial_fit())

for i in range(num_samples):

    estimator.fit(X[i], y[i])

    estimator.coeff_ OR estimator.intercept_

Hope it clears up and give you a way to do it. Feel free to ask if need help.