CODEWITHSUNDEEP
gunicornfastapiuvicorn
Jerry
Jerry

Reputation: 414

how many uvicorn workers do I have to have in production?

My Environment

Document

https://fastapi.tiangolo.com/deployment/server-workers/

Question

Currently I'm using 24 Uvicorn workers in production server. (c5.2xlarge)

gunicorn main:app --workers 24 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:80

I've learn that one process runs on one core. Therefore If i have 8 processes, I can make use of whole cores (c5.2xlarge's vCpu == 8)

I'm curious that in this situation, Is there any performance benefit if I got more processes than 8?

Upvotes: 18

Views: 47194

Answers (2)

John S John
John S John

Reputation: 422

Number of recommended workers is 2 x number_of_cores +1

You can more about it in the official documentation.

In your case with 8 CPU cores, you should be using 17 worker threads.

Additional thoughts on async systems:

The two times core is not a scientific figure as says in the article. But the idea is that one thread can do I/O and another CPU processing at the same time. This makes maximum use of simultaneous threads. Even with async systems, conceptually this works and should give you maximum efficiency.

Upvotes: 19

MikeL
MikeL

Reputation: 5611

In general the best practice is:

number_of_workers = number_of_cores x 2 + 1

or more precisely:

number_of_workers = number_of_cores x num_of_threads_per_core + 1

--> BUT be careful not to confuse number_of_cores with vCPU.

The reason for it is CPU hyperthreading, which allows each core to run multiple concurrent threads. The number of concurrent threads is decided by the chip designers.

Two concurrent threads per CPU core are common, but some processors can support more than two.

vCPU that is mentioned for AWS ec2 resource is already the hyperthreaded amount of processing units you have on the machine (num_of_cores x num_of_threads_per_core). Not to be confused with number_of_cores available on that machine.

So, in your case, c5.2xlarge has 8 vCPUs, meaning you have 8 available concurrent workers.

Upvotes: 13

Related Questions