CODEWITHSUNDEEP
pythonmultithreadinggrpcgrpc-python
Tushar Seth
Tushar Seth

Reputation: 613

grpc-Python max_workers limiting number of simultaneous processes

while using python grpc server,

server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

this is general way in which grpc server is instantiated. But with this running, if I try to run more than 10 instances of client , which expects server streaming, the 11th one doesn't work (I am running 10 instances of client which connects to this server and gets the stream)

Even if I change max_workers to None, max it creates is 40 threads (8 cores x 5 as per documentation), so max 40 clients can be served simultaneously in that case.
Is this the expected behavior ?

I was working on my code, but tried with general grpc python code documented here:

https://grpc.io/docs/tutorials/basic/python.html

I am able to reproduce the same issue with this.

To reproduce it, just run route_guide_server.py in one window with max_workers= 4 and then try to run 4-5 different clients in different windows . The 4th client will have to wait till one of the client is finished. (To get better view, add a time.sleep in yield)

If a large number of clients (100s and 1000s of clients) want to access grpc server in python with streaming (which should be continuous), then anymore clients will never get chance.

Upvotes: 1

Views: 6695

Answers (1)

Kevin S
Kevin S

Reputation: 960

Yes this is the expected behavior.

After running my own test code, yes if you supply an argument of None to max_workers then 40 is the max. However, if I set the max to 100 then sure enough I can have at most 100 concurrent workers. This should be expected behavior because a thread pool is created based on the number of workers requested. You cannot expect that if you don't supply a number of max workers that it will just scale up and down at run time. Not without changing grpc and concurrent futures thread pool. With the way the interface is coupled, in python grpc right now we must used concurrent futures threadpool, so we must supply an argument to max_workers if we want it to be more than 40, and it must be set at compile time.

Upvotes: 1

Related Questions