CODEWITHSUNDEEP
python-3.xamazon-web-servicestensorflow-servingamazon-sagemaker
Austin
Austin

Reputation: 7319

SageMaker Script Mode Serving

I've trained a tensorflow.keras model using SageMaker Script Mode like this:

import os

import sagemaker
from sagemaker.tensorflow import TensorFlow

estimator = TensorFlow(entry_point='train.py',
                       source_dir='src',
                       train_instance_type=train_instance_type,
                       train_instance_count=1,
                       hyperparameters=hyperparameters,
                       role=sagemaker.get_execution_role(),
                       framework_version='1.12.0',
                       py_version='py3', 
                       script_mode=True)

However, how do I specify what the serving code is when I call estimator.deploy()? And what is it by default? Also is there any way to modify the nginx.conf using Script Mode?

Upvotes: 2

Views: 1492

Answers (2)

Rui
Rui

Reputation: 61

With script mode the default serving method is the TensorFlow Serving-based one: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/estimator.py#L393 Custom script is not allowed with the TFS based container. You can use serving_input_receiver_fn to specify how the input data is processed as described here: https://www.tensorflow.org/guide/saved_model

As for modifying the ngnix.conf, there are no supported ways of doing that. Depends on what you want to change in the config file you can hack the sagemaker-python-sdk to pass in different values for these environment variables: https://github.com/aws/sagemaker-tensorflow-serving-container/blob/3fd736aac4b0d97df5edaea48d37c49a1688ad6e/container/sagemaker/serve.py#L29

Here is where you can override the environment variables: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/serving.py#L130

Upvotes: 1

Julien Simon
Julien Simon

Reputation: 2719

The Tensorflow container is open source: https://github.com/aws/sagemaker-tensorflow-container You can view exactly how it works. Of course, you can tweak it, build it locally, push it to ECR and use it on SageMaker :)

Generally, you can deploy in two ways:

I would also recommend looking at the TensorFlow examples here: https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk

Upvotes: 4

Related Questions