CODEWITHSUNDEEP
tensorflowtensorflow-serving
Harel Farkash
Harel Farkash

Reputation: 166

Tensorflow serving using the client

I successfully created a server that receives a TF saved_model, but now I want to send it queries and get predictions. However, I'm having a hard time of understanding how the client works and how to implement it. All I found online is the basic tutorial, but they only give the client code for mnist, and it doesn't fit my own mdoel. So can anyone refer me to how to use or implement the client for a different model?

Thanks

Upvotes: 1

Views: 519

Answers (1)

Xinyao Wang
Xinyao Wang

Reputation: 2159

I really thank google to make tensorflow serving open source, it is so helpful for people like me to put prediction models into production. But I have to admit tensorflow serving did poorly in documentation, or, they assume people who use it should already have pretty good knowledge in tensorflow. I stuck for a long time in order to get understand how it works. In their website they introduced concepts and examples well, but there is something missing in between.

I will recommend the tutorial here. This is the first part, and you can also follow the second part, the link will be in that article.

In general, when you export your .ckpt files to a servable model(.pb file and variables folder), you have to define the input, output and method name of your model and save them as a signature in tf.saved_model.signature_def_utils.build_signature_def

In the article, you will find what I said above in this part:

    tf.saved_model.signature_def_utils.build_signature_def( 
    inputs={‘images’: predict_tensor_inputs_info}, 
    outputs={‘scores’: predict_tensor_scores_info},
    method_name=\
        tf.saved_model.signature_constants.PREDICT_METHOD_NAME)

You can follow how the author defined input and output in the article, and do the same thing to your customized model.

After that, you have to in your client script to call the signature and feed input to server, server then will recognize which method to use and return the output. You can check how the author wrote the client script and find corresponding part of calling signature and feeding input.

Upvotes: 3

Related Questions