CODEWITHSUNDEEP
scikit-learnpipelinegcloudgoogle-cloud-ml
NickDGreg
NickDGreg

Reputation: 484

Can a custom sklearn model be uploaded to gcloud ml-engine?

I have tried out the new functionality to deploy an sklearn model to ml-engine following the gcloud tutorial. I have created a wrapper for the sklearn Estimator GaussianMixture in order to alter the predict function so that it outputs probabilistic outputs (like sample_scores) instead of the default class predictions.

GMMWrapper.py

class GMMWrapper(GaussianMixture):
    def predict(self, X):
        return self.score_samples(X)

I create the pipeline and save it in a standard way:

gmm = GMMWrapper()
pipegmm = Pipeline([('gmm', gmm)])
pipegmm.fit(data)
with tf.gfile.Open(model_output_path, 'wb') as model_file:
    joblib.dump(pipegmm, model_output_path + '.joblib')

However when deploying to gcloud using gcloud beta ml-engine versions create of course it cannot find the module GMMWrapper

ERROR: (gcloud.beta.ml-engine.versions.create) Bad model detected with
error:  "Failed to load model: Could not load the model: 
/tmp/model/0001/model.joblib. No module named 'GMMWrapper'.

I have tried to include it in the deployment source folder which will be uploaded to the staging bucket.

DEPLOYMENT_SOURCE="/folder-containing-model-and-GMMWrapper"
MODEL_NAME="GMMs"
FRAMEWORK="SCIKIT_LEARN"
STAGING_BUCKET="gs://sklearn-models"

gcloud beta ml-engine versions create v1 \
    --model $MODEL_NAME --origin $DEPLOYMENT_SOURCE \
    --framework $FRAMEWORK --staging-bucket $STAGING_BUCKET \
    --runtime-version=1.5 --python-version=3.5

But that returns the same error. I would need to somehow inform gcloud where to look for the module. I suspect this cannot be done, but thought I would ask in case there is a way.

(Alternatively, if theres a way to change which function is called when calling predict on the pipeline that could also solve my problem.)

Upvotes: 1

Views: 369

Answers (1)

rhaertel80
rhaertel80

Reputation: 8399

This functionality is not currently supported. We're always seeking ways to improve the product and will take this suggested feature into account.

Upvotes: 1

Related Questions