CODEWITHSUNDEEP
tensorflowtensorflow-serving
user1371314
user1371314

Reputation: 832

Tensorflow Serving predictions mapped to labels

I am serving up the inception model using TensorFlow serving. I am doing this on Azure Kubernetes so not via the more standard and well documented google cloud.

In any event, this is all working however the bit i am confused about is the predictions come back as an array of floats. These values map to the original labels passed in during training but without the original labels file there is no way to reverse engineer what each probability relates to.

Before I moved to serving i was simply using an inference script that then cross references against the labels file which i stored along with the frozen model at time of training. But with serving this does not work.

So my question is how can i get the labels associated with the model and ideally get the prediction to return the labels and probabilities?

Upvotes: 4

Views: 1876

Answers (2)

user 007
user 007

Reputation: 911

i tried the approach suggested by @user1371314 but i couldn't get it to work. An other solution that worked is creating a tensor (instead of a constant) and map it with only the first element of the output layer when saving the model. When you put it together it looks like this :

# get labels names and create a tensor from it
.... 
label_names_tensor = tf.convert_to_tensor(label_names)

# save the model and map the labels to the output layer
tf.saved_model.simple_save(
        sess,
        "./saved_models",
        inputs={'image': model.input},
        outputs={'label' : label_names_tensor,'prediction': model.output[0]})

When you make a prediction after serving your model you will get the following result:

{
    "predictions": [
        {
            "label": "label-name",
            "prediction": 0.114107
        },
        {
            "label": "label-name",
            "prediction": 0.288598
        },
        {
            "label": "label-name",
            "prediction": 0.17436
        },
        {
            "label": "label-name",
            "prediction": 0.186366
        },
        {
            "label": "label-name",
            "prediction": 0.236568
        }
    ]

Upvotes: 2

user1371314
user1371314

Reputation: 832

I am sure there is a way to return a mapping directly for this using the various TF ops however I have managed to at least package the labels into the model and return them in the prediction along with the probabilities.

What i did was create a tf.constant from the labels array and then added that tensor to the array of output tensors in tf.saved_model.signature_def_utils.build_signature_def

Now when i get a prediction i get the float array and also an array of labels and i can match them up on the client side.

Upvotes: 1

Related Questions