CODEWITHSUNDEEP
resttensorflowgoogle-cloud-platformcomputer-visionvirtual-machine
Gokul Krishnan R
Gokul Krishnan R

Reputation: 83

Tensorflow Serving on a remote VM to serve HTTP prediction request from clients

As per the official Tensorflow Serving Documentation here, I have created the Model service on a remote VM with GPU to serve on localhost and prediction works just fine. However, the intention is to serve the model on remote VM, such that the clients would be able to send the HTTP prediction requests to this remote server whose IP address is known to them. On a client machine, I changed 'localhost' to the remote IP keeping everything else same and sent the request. However, the connection times out. Any help / suggestion is highly appreciated.
curl -d '{"instances": [1.0,2.0,5.0]}' -X POST http://<Remote Machine IP>:8501/v1/models/saved_model_half_plus_three:predict

Upvotes: 0

Views: 216

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75920

The port is not open. Go the the firewall rules and add the port 8501 from 0.0.0.0/0 source IP range, if it's what you want.

But, for production, I recommend you to put this behind a load balancer, to have a manage instance group and other scalable and secure stuff.

Upvotes: 2

Related Questions