How to deploy Pytorch in Python via a REST API with Flask?

Question

I am working on AWS Sagemaker and my goal is to follow this tutorial from Pytorch's official documentation.

The original predict function from the tutorial above is the following:

@app.route('/predict', methods=['POST'])
def predict():
    if request.method == 'POST':
        file = request.files['file']
        img_bytes = file.read()
        class_id, class_name = get_prediction(image_bytes=img_bytes)
        return jsonify({'class_id': class_id, 'class_name': class_name})

I was getting this error, so I added 'GET' as a method as mentioned in here. I also simplified my example to its minimal expression:

from flask import Flask, jsonify, request

app = Flask(__name__)

@app.route('/predict', methods=['GET','POST'])
def predict():
    if request.method == 'POST':
        return jsonify({'class_name': 'cat'})
    return 'OK'

if __name__ == '__main__':
    app.run()

I perform requests with the following code:

import requests

resp = requests.post("https://catdogclassifier.notebook.eu-west-1.sagemaker.aws/proxy/5000/predict",
                     files={"file": open('/home/ec2-user/SageMaker/cat.jpg', 'rb')})

resp is but resp.json() returns JSONDecodeError: Expecting value: line 1 column 1 (char 0) Finally, resp.url points me to a page saying 'OK'.

Moreover, this is the output of resp.content

What am I missing?

shrmaab · Accepted Answer

Looks like the content of your resp is HTML as opposed to JSON; this is likely a consequence of how the Jupyter server proxy endpoint you're attempting to POST to (https://catdogclassifier.notebook.eu-west-1.sagemaker.aws/proxy/5000/predict) is configured.

It looks like you're using a SageMaker notebook instance, so you might not have much control over this configuration. A workaround could be to instead deploy your Flask server as a SageMaker endpoint running outside JupyterLab, instead of directly on a notebook instance.

If you want to prototype using only a notebook instance, you can alternately just bypass the proxy entirely and simply call your Flask route relative to localhost from another notebook tab while the Flask server runs in your main notebook tab:

import requests

resp = requests.post("https://localhost:5000/predict",
                     files={"file": open('/home/ec2-user/SageMaker/cat.jpg', 'rb')})

How to deploy Pytorch in Python via a REST API with Flask?

Answers (1)

Related Questions