CODEWITHSUNDEEP
pickletheanoprediction
Filippo Galli
Filippo Galli

Reputation: 61

Save Theano model doens't work for MLP net

I'm trying to save a model obtained during training of a Multi layer Perceptron network built with Theano according to http://deeplearning.net/tutorial/code/mlp.py using the code shown in the Logistic regressor at http://deeplearning.net/tutorial/code/logistic_sgd.py, in particular

# save the best model
with open('best_model.pkl', 'w') as f:
cPickle.dump(classifier, f)

but what I get is

... loading data ... building the model ... training epoch 1, minibatch 74/74, validation error 38.333333 % epoch 1, minibatch 74/74, test error of best model 41.666667 % Traceback (most recent call last): File "mlp.py", line 423, in test_mlp() File "mlp.py", line 406, in test_mlp cPickle.dump(classifier, f, protocol=cPickle.HIGHEST_PROTOCOL) cPickle.PicklingError: Can't pickle : attribute lookup builtin.instancemethod failed

Since I met this problem also with the convolutional network, my question is: there is a general way to store a model in Theano, to be reused for prediction?

EDIT As suggested in the comments I'm using now 

cPickle.dump((classifier.hiddenLayer.params,classifier.logRegressionLayer.params), f)

for saving and 

classifier.hiddenLayer.W = cPickle.load(open('best_model_mlp.pkl'))[0][0]

for setting the weights (for instance) of the hiddenLayer in the classifier defined as

x = T.matrix('x')
classifier = MLP(
    rng=rng,
    input=x,
    n_in = 28*28,
    n_hidden= 500,
    n_out=10
)

but when I'm calling the very function

predict_model = theano.function(
    inputs=[classifier.input],
    outputs=classifier.logRegressionLayer.y_pred,
    )

I always have [0] as a prediction, even with a well trained net. I'm still doing wrong in setting or saving parameters?

Upvotes: 4

Views: 563

Answers (2)

Fla
Fla

Reputation: 9

I also ran into this problem and found this solution. Even if only 'classifier.params' should be necessary, y_pred and input also need to be initialized. Simple way is to store them via pickle and reload them.

Saving:

with open('best_model.pkl', 'wb') as f:
    cPickle.dump((classifier.params, classifier.logRegressionLayer.y_pred, 
                 classifier.input), f)

Predicting function:

def predict(dataset, n_hidden, n_in, n_out):
    datasets = load_data(dataset)
    test_set_x, test_set_y = datasets[2]
    test_set_x = test_set_x.get_value()
    test_set_y = test_set_y.eval()

    rng = numpy.random.RandomState(1234)
    x = T.matrix('x')

    # Declare MLP classifier
    classifier = MLP(
        rng=rng,
        input=x,
        n_in=n_in,
        n_hidden=n_hidden,
        n_out=n_out
    )

    # load the saved model
    classifier.params, classifier.logRegressionLayer.y_pred,
        classifier.input = cPickle.load(open('best_model.pkl'))

    predict_model = theano.function(
        inputs=[classifier.input],*emphasized text*
        outputs=classifier.logRegressionLayer.y_pred)

    print("Expected values: ", test_set_y[:10])
    predicted_values = predict_model(test_set_x[:10])
    print("Predicted values:", predicted_values)

Hope this helps.

Upvotes: 0

Cazoulable
Cazoulable

Reputation: 51

I've been facing the same problem today. I don't know exactly the reason why this works with 'logistic_sgd.py' and not with 'mlp.py' but what you can do is to just store 'classifier.params'

'classifier.params' are the only elements you need once you're done with the training. It should not be too hard to predict your classes from this parameters (this is already in your code).

Upvotes: 3

Related Questions