CODEWITHSUNDEEP
pythonflaskgunicorn
Swapnil
Swapnil

Reputation: 191

Handle 1000 concurrent requests for Flask/Gunicorn web service

I am fairly new to creating web services in Python. I have created a Flask web service successfully and run it with Gunicorn (as Flask’s built-in server is not suitable for production). This is how I run my app (with 4 worker nodes).

   gunicorn --bind 0.0.0.0:5000 My_Web_Service:app -w 4

The problem is, this only handles 4 requests at a time. I want it to be able to handle potentially 1000's of requests concurrently. Should I be using multi-threading? Any other options/suggestions?

Upvotes: 8

Views: 14261

Answers (2)

mirekphd
mirekphd

Reputation: 6743

I'd switch from Flask to FastAPI and combine it with either Async IO or (if it's not possible to find non-blocking versions for all your functions) with a multiprocessing pool (not multithreading, which would be still blocked by the GIL and thus slightly slower).

Among production servers gunicorn is probably still the best process manager, but since FastAPI needs ASGI, you need to combine it with uvicorn workers.

Upvotes: 0

Daniel
Daniel

Reputation: 42748

Reading the section on Workers you have to switch to an async worker, which can handle thousands of connections, if your work is IO bound. Using more processes than CPUs is not recommended.

Upvotes: 6

Related Questions