CODEWITHSUNDEEP
grpctensorflow-serving
Abhisek
Abhisek

Reputation: 5244

Servable not found for request: while loading an older version of the model

I've a model, let's say mymodel and two different data sets: setA, setB.

After training(in my local machine) setA and setB separately, tensorflow serving created two different directories: 100, 200 for setA and setB respectively.

To host the model inside docker

root@ccb58054cae5:/# ls /serving/model/
100 200
root@ccb58054cae5:/# bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=mymodel --model_base_path=/serving/model &> log &

Now when I do the inference for setB, I'm successfully able to get the response, as tensorflow serving by default loads 200 because it thinks this is the latest model.

Now I want to query for setA, so I need to mention in the code which version of the hosted model to hit and that would be 100.

In terms of code: request.model_spec.version.value = 100

for completeness here is the other relevant client code:

host, port = FLAGS.server.split(':')
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = 'mymodel'
request.model_spec.signature_name = signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY
request.model_spec.version.value = 100

I got to know about request.model_spec.version.value = 100 from here. But I got unlucky, I'm getting:

Traceback (most recent call last):
  File "C:\Program Files\Anaconda3\lib\site-packages\grpc\beta\_client_adaptations.py", line 193, in _blocking_unary_unary
    credentials=_credentials(protocol_options))
  File "C:\Program Files\Anaconda3\lib\site-packages\grpc\_channel.py", line 492, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "C:\Program Files\Anaconda3\lib\site-packages\grpc\_channel.py", line 440, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.NOT_FOUND, Servable not found for request: Specific(mymodel, 100))>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Anaconda3\lib\site-packages\flask\app.py", line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Program Files\Anaconda3\lib\site-packages\flask\app.py", line 1641, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Program Files\Anaconda3\lib\site-packages\flask_cors\extension.py", line 188, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Program Files\Anaconda3\lib\site-packages\flask\app.py", line 1544, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Program Files\Anaconda3\lib\site-packages\flask\_compat.py", line 33, in reraise
    raise value
  File "C:\Program Files\Anaconda3\lib\site-packages\flask\app.py", line 1639, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Program Files\Anaconda3\lib\site-packages\flask\app.py", line 1625, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "mymodel.py", line 63, in json_test
    response =  main.main(ser, query = que)
  File "D:\mymodel_temp\temp\main.py", line 23, in main
    return json_for_inference(model.inference(query), query, service_id)
  File "D:\mymodel_temp\temp\src\utils.py", line 30, in wrapper
    outputs = function(self, *args, **kwargs)
  File "D:\mymodel_temp\temp\src\model.py", line 324, in inference
    result = stub.Predict(request, 10.0) # 10 seconds
  File "C:\Program Files\Anaconda3\lib\site-packages\grpc\beta\_client_adaptations.py", line 309, in __call__
    self._request_serializer, self._response_deserializer)
  File "C:\Program Files\Anaconda3\lib\site-packages\grpc\beta\_client_adaptations.py", line 195, in _blocking_unary_unary
    raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.NOT_FOUND, details="Servable not found for request: Specific(mymodel, 100)")

Upvotes: 1

Views: 2799

Answers (1)

Dharmendra
Dharmendra

Reputation: 308

You are getting Servable not found for request: Specific(mymodel, 100) error, because there is no model loaded with this specific version number.

root@ccb58054cae5:/# bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=mymodel --model_base_path=/serving/model &> log &

When you run above command on docker for serving models, it will load model according to Version Policy mentioned here in Git Code of Tensorflow Serving. By default only one version of model will be loaded and that version will be Latest one (Higher version number).

If you want to load multiple versions or multiple models, you need to add --model_config_file flag. Also remove --model_name and --model_base_path flags as we will be adding them in model config file.

So now your command will be like this :-

root@ccb58054cae5:/# bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_config_file=mymodel.conf &> log &

Format of model config file will be like this :-

model_config_list: {

    config: {
        name: "mymodel",
        base_path: "/serving/model",
        model_platform: "tensorflow"
        model_version_policy: {all{}}
    }
}

So you can set Version Policy other than latest (By Default) to all and specific. By selecting all , you can load all versions of model as servables and then you can easily access specific version using request.model_spec.version.value = 100 in client code. If you want to load more than one models, you can do that in same config file by appending model config for those models.

Upvotes: 5

Related Questions