CODEWITHSUNDEEP
pythontensorflowrequestkerasgoogle-cloud-ml
Anish Khanna
Anish Khanna

Reputation: 101

How to deploy a Keras CNN Model to webservice?

I am currently in the process of trying to deploy a Keras Convolutional Neural Network for a webservice.

I had tried converting my saved keras hdf5 model to a tensorflow.js model and deploying that but it slowed down the client-side app as the model is relatively robust and thus, takes a large amount of space in the client memory.

Thus, I am trying to figure out a way to deploy the model in the cloud and make predictions through a request from the web-app with an image, and then receive a response which holds the prediction tensor. I know that gcloud may have some similar abilities or feature but I am unsure of how to get started.

Essentially, I am asking if there is any service that will allow me to deploy a pre-trained and saved convolutional neural network model to which I can send images in a request and use the model to return a predicted tensor?

Upvotes: 2

Views: 1824

Answers (2)

sdcbr
sdcbr

Reputation: 7129

You can export a trained Keras model and serve it with TensorFlow Serving. TF Serving allows to host models and call them via either gRPC or REST requests. You could deploy a flask app with an endpoint that accepts an image, wraps it as a payload and calls your model via the requests module.

Your code to export the model as a servable would look like this:

import tensorflow as tf

# The export path contains the name and the version of the model
model = keras.models.load_model('./mymodel.h5')

# Feth the Keras session and save the model
with keras.backend.get_session() as sess:
    tf.saved_model.simple_save(
        sess,
        export_dir,
        inputs={'images': model.input},
        outputs={t.name:t for t in model.outputs})

This will store the files necessary for TF Serving. From this directory, you can host the model as follows:

tensorflow_model_server --model_base_path=$(pwd) --rest_api_port=9000 --model_name=MyModel

Your request would then look like this:

requests.post('http://ip:9000/v1/models/MyModel:predict', json=payload)

Where payload is a dictionary that contains your request image.

Upvotes: 2

rhaertel80
rhaertel80

Reputation: 8389

If you want a click-to-deploy solution for serving your model on Google Cloud, consider using Cloud ML Engine's Online Prediction service. First, follow the instructions in @sdcbr's response to export your SavedModel. Copy the model to GCS then you simply create a model and a version:

gcloud ml-engine models create "my_image_model"
gcloud ml-engine versions create "v1"\
    --model "my_image_model" --origin $DEPLOYMENT_SOURCE

Or, even easier, use Cloud Console to do the above with a few clicks!

You will get a serverless REST endpoint that includes authentication and authorization, autoscaling (including scale to zero) as well as logging and monitoring, without having to write or maintain a line of code.

Upvotes: 1

Related Questions