TensorFlow Serving: How to get prediction, ModelSpec missing

Question

I created a simple Multi-Layer-Perceptron to test TensorFlow Serving. It simply takes two numbers and should predict the sum of those two numbers. Here is the code that I wrote:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import os
import sys
import logging
import numpy as np
import tensorflow as tf

tf.app.flags.DEFINE_integer("model_version", 1, "version number of the model.")
tf.app.flags.DEFINE_string("work_dir", "/tmp", "Working directory.")
FLAGS = tf.app.flags.FLAGS

def main():

    # DATA PREPARATION
    # ================================================
    data_train = np.array(
        [[1, 3], [2, 5],
         [3, 1], [3, 3],
         [4, 2], [7, 1],
         [8, 1], [2, 2],
         [5, 1], [1, 7],
         [0, 1], [0, 5],
         [0, 7], [0, 8],
         [1, 1], [1, 2],
         [0, 0], [1, 8]]
    )

    labels_train = np.array(
        [4, 7, 4, 6, 6, 8, 9, 4, 6, 8, 1, 5, 7, 8, 2, 3, 0, 9]
    )

    # specify that all features have real-value data
    feature_columns = [tf.feature_column.numeric_column("x", shape=[2])]

    # CREATE CLASSIFIER
    # ================================================
    # build 3 layer dnn with 10, 20, 10 units respectively.
    classifier = tf.estimator.DNNClassifier(
        feature_columns=feature_columns,
        hidden_units=[10, 20, 10],
        n_classes=10,
        model_dir="/tmp/addition_model"
    )

    # TRAINING PHASE
    # ================================================
    # define the training inputs
    train_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"x": data_train},
        y=labels_train,
        num_epochs=None,
        shuffle=True
    )

    # train model
    classifier.train(input_fn=train_input_fn, steps=10000)

    # PREDICT NEW SAMPLES (LOCAL)
    # ================================================
    # new_samples = np.array(
    #     [[0, 6]],
    #     dtype=np.float32
    # )
    #
    # predict_input_fn = tf.estimator.inputs.numpy_input_fn(
    #     x={"x": new_samples},
    #     num_epochs=1,
    #     shuffle=False
    # )
    #
    # predictions = list(classifier.predict(input_fn=predict_input_fn))
    # predicted_classes = [p["classes"] for p in predictions]
    #
    # print("Predictions: {}".format(predicted_classes))

    # BUILD AND SAVE MODEL
    # ================================================
    export_path_base = sys.argv[-1]
    export_path = os.path.join(
        tf.compat.as_bytes(export_path_base),
        tf.compat.as_bytes(str(FLAGS.model_version))
    )

    feature_spec = {"x": tf.FixedLenFeature([2], tf.float32)}

    def serving_input_receiver_fn():
        serialized_tf_example = tf.placeholder(dtype=tf.string,
                                           shape=[None],
                                           name="input_tensors")
        receiver_tensors = {"inputs": serialized_tf_example}
        features = tf.parse_example(serialized_tf_example, feature_spec)

        return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

    classifier.export_savedmodel(export_path, serving_input_receiver_fn)

if __name__ == "__main__":
    logging.getLogger().setLevel(logging.INFO)
    main()

This code workes perfectly fine and it gives me predictions. As a next step I exported the model and deployed it on the TensorFlow Serving Server (which also works). I face some issues with the client code:

from grpc.beta import implementations
from tensorflow_serving.apis import prediction_service_pb2
from tensorflow_serving.apis import classification_pb2

host_port = "localhost:9000"
host, port = host_port.split(":")
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)

request = classification_pb2.ClassificationRequest()
example = request.input.example_list.examples.add()
example.features.feature["x"].float_list.value.extend([2, 6])

result = stub.Classify(request, 10.0)  # 10 secs timeout
print(result)

In this case I would like to predict the sum of 2 + 6. But it keeps telling me that a ModelSpec is missing when calling stub.Classify(request, 10.0), but I have no idea how to specify it (and I could not find anything in the documentation):

grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="Missing ModelSpec")

Does anybody have an idea? I am also not sure whether the exporting part is correct... Any suggestions for improvement are highly appreciated.

Thank you very much for your support

Answer 1

I have just found the solution, I was kind of blind :D

Sometimes it is as simple as that:

request.model_spec.name = "addition"