CODEWITHSUNDEEP
javamultithreadingtomcatgrpcgrpc-java
Chung
Chung

Reputation: 3

Multiple requests go into same thread of Java gRPC Server

I have some micro services written by java, communicating with each other by gRPC, and are put into AWS ECS for running. When I run the service and use a client program to call these services, I have noticed a very strange thing.

Normally, Tomcat's thread pool setting will make sure multiple requests will be handled by same amount of threads. But when I run my program sometimes it will happen a thread start handling a new request when it is already handling a previous request. It makes the process thread-unsafe and thus errors were happened. It happens very rare on normal services, and must happens when I send 10 requests into same service at the same time.

May I ask is there any setting that I have missed? What I can search on the internet is I can only increase thread pool size.

----------2020/12/22 Edited----------

By entering debug mode, I have seen that one RPC request will be split to two tasks. One for message itself and another for listener returned from the first task. 90% of my previous errors were happened between the time gap of these two tasks. Moreover, when running listener, ServerImpl class will use context to handle each RPC message, and each step in unit of methods that covers all written codes within over-rided listeners. Therefore it will happen thread-switching during listener's run.

Upvotes: 0

Views: 3520

Answers (1)

Eric Anderson
Eric Anderson

Reputation: 26464

In StreamObserver's documentation:

Implementations are not required to be thread-safe (but should be thread-compatible).

The "thread-compatible" is the important part for your question. It means you can't assume that every call to your instance will happen on the same Thread. This is true for virtually all of the gRPC API that is not-thread-safe. That doesn't matter to most code, but would impact things like ThreadLocals.

Basically, a thread is not "owned" by an RPC. Instead, the RPCs share all the threads and when a callback needs to be delivered it just finds a thread to run the callback on. This an async model that allows greater scaling compared to the mostly-blocking model used with servlets, as you can service more RPCs with fewer threads. But it means as soon as you return from a callback that same thread may be used for some other RPC callback.

Upvotes: 2

Related Questions