CODEWITHSUNDEEP
performancemachine-learningscikit-learncomputer-vision
Su JK
Su JK

Reputation: 171

Slow prediction: Scikit Gaussian Process classification

Following is the code to evaluate to train a Gaussian Process (GP) and use to classify the images from MNIST dataset.

import numpy as np


from sklearn.metrics.classification import accuracy_score, log_loss
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.gaussian_process.kernels import RBF
from sklearn import datasets
from sklearn.datasets import fetch_mldata

import random

SAMPLE_SIZE = 2000

def build_and_optimize(hp_space):
    build_train()
    return

def build_train(hp_space):
    l_scale = hp_space['l_scale']
    bias = hp_space['bias']
    gp_fix = GaussianProcessClassifier(kernel=bias * RBF(length_scale=l_scale), optimizer=None, multi_class='one_vs_rest')

    X_train, X_test, y_train, y_test = prepare_data()


    gp_fix.fit(X_train, y_train)

    print("Log Marginal Likelihood (initial): %.3f"
        % gp_fix.log_marginal_likelihood(gp_fix.kernel_.theta))


    y_ = gp_fix.predict(X_test[0:500])
    print(y_)
    print(y_test[0:500])
    print("Accuracy: %.3f (initial)"
        % (accuracy_score(y_test[0:500], y_)))

    return


def prepare_data():
    mnist = fetch_mldata('MNIST original', data_home='./')
    mnist.keys()

    images = mnist.data
    targets = mnist.target

    X_data = images/255.0
    Y = targets
   shuf = random.sample(range(len(X_data)), SAMPLE_SIZE)

    X_train = []
    for x in shuf: X_train.append(X_data[x])

    y_train = []
    for x in shuf: y_train.append(Y[x])

    c_shuf = set(range(len(X_data))) - set(shuf)

    X_test = []
    for x in c_shuf: X_test.append(X_data[x])

    y_test = []
    for x in c_shuf: y_test.append(Y[x])

    return X_train, X_test, y_train, y_test

if __name__ == "__main__":
    hp_space = {}
    hp_space['l_scale'] = 1.0
    hp_space['bias'] = 1.0

    build_train(hp_space)

It seems the training of the model takes decent amount of time. However, the prediction takes ages. Any pointers as what could be the reason?

Upvotes: 2

Views: 5689

Answers (2)

Raff.Edward
Raff.Edward

Reputation: 6514

You can think of Gaussian processes and SVMs are somewhat similar models, both do use the kernel trick to build a model. Lik SVMs, GPs take O(n^3) time to train, where n is the number of data points in the training set. Thus you should naturally expect it to take a while to train, and for it to grow quickly as you increase the dataset size.

Similarly, the GP prediction takes O(n) time per prediction, similar to a nearest neighbor search and SVMS. However, the GP solution is dense, meaning it uses all the training points for prediction - where the SVM is sparse, so it gets to throw some away.

Upvotes: 7

seralouk
seralouk

Reputation: 33147

You could speed up the process by addind n_jobs = -1 as follows:

gp_fix = GaussianProcessClassifier(kernel=bias * RBF(length_scale=l_scale), optimizer=None, multi_class='one_vs_rest', n_jobs = -1)

This way from the documentation:

The number of jobs to use for the computation. If -1 all CPUs are used. If 1 is given, no parallel computing code is used at all, which is useful for debugging. For n_jobs below -1, (n_cpus + 1 + n_jobs) are used. Thus for n_jobs = -2, all CPUs but one are used.

Source: here

Upvotes: 2

Related Questions