CODEWITHSUNDEEP
python-3.xtensorflowtensorflow-servingtensorflow-hub
Surya Pratap Singh
Surya Pratap Singh

Reputation: 21

TF - hub universal sentence encoder module save and reload?

I am using universal sentence encoder pre-trained model using below command:

import tensorflow as tf
import tensorflow_hub as hub

MODEL_NAME = 'tf-sentence-encoder'
VERSION = 1
SERVE_PATH = './models/{}/{}'.format(MODEL_NAME, VERSION)

with tf.Graph().as_default():
  module = hub.Module("https://tfhub.dev/google/universal-sentence-encoder-large/3")
  text = tf.placeholder(tf.string, [None])
  embedding = module(text)

  init_op = tf.group([tf.global_variables_initializer(),
                      tf.tables_initializer()]
                     )
  with tf.Session() as session:
      session.run(init_op)
      tf.saved_model.simple_save(session, SERVE_PATH,
                                 inputs = {"text": text}, outputs = {"embedding": embedding},
                                 legacy_init_op = tf.tables_initializer()
                                 )

How to reload the saved model for RESTFUL APIs?

Upvotes: 2

Views: 600

Answers (1)

user11530462
user11530462

Reputation:

As Arno mentioned in the comment, this question is related to Tensorflow Serving.

Once you Save the Model using tf.saved_model.simple_save, the model will be Saved in .pb format.

By reloading the Saved Model using REST API, I understand performing Inference using REST API. It is explained below:

Let's consider the Model Name as tf_sentence_encoder You can view the SignatureDef of your model using the below command:

!saved_model_cli show --dir tf_sentence_encoder --all

We should Install Tensorflow Serving using Docker Pull Command shown below:

sudo docker pull tensorflow/serving

Then we have to Invoke the Tensorflow Model Server:

sudo docker run -p 8501:8501 --mount type=bind,source=/Path_Of_The_Model/tf_sentence_encoder,target=/models/tf_sentence_encoder -e MODEL_NAME=tf_sentence_encoder -t tensorflow/serving &

Then, you can perform the Prediction using REST API as shown below:

curl -d '{"inputs": [5.1,3.3,1.7,0.5]}' \ -X POST http://localhost:8501/v1/models/tf_sentence_encoder:predict

Upvotes: 1

Related Questions