Java's Spring Boot vs Python's FastAPI: Threads

Question

I'm a Java Spring boot developer and I develop 3-tier crud applications. I talked to a guy who seemed knowledgeable on the subject, but I didn't get his contact details. He was advocating for Python's FastAPI, because horizontally it scales better than Spring boot. One of the reasons he mentioned is that FastAPI is single-threaded. When the thread encounters a database lookup (or other work the can be done asyncly), it picks up other work to later return to the current work when the database results have come in. In Java, when you have many requests pending, the thread pool may get exhausted.

I don't understand this reasoning a 100%. Let me play the devil's advocate. When the Python program encounters an async call, it must somehow store the program pointer somewhere, to remember where it needs to continue later. I know that that place where the program pointer is stored is not at all a thread, but I have to give it some name, so let's call it a "logical thread". In Python , you can have many logical threads that are waiting. In Java, you can have a thread pool with many real threads that are waiting. To me, the only difference seems to be that Java's threads are managed at the operating system level, whereas Python's "logical threads" are managed by Python or FastAPI. Why are real threads that are waiting in a thread pool so much more expensive than logical threads that are waiting? If most of my threads are waiting, why can't I just increase the thread pool size to avoid exhaustion?

Answer 1

Python has GIL, which makes the system to run only 1 thread per time (you could make multiple threads, but it will only run a single thread even if you run it on multi-core architecture)

To overcome this issue, tools like gunicorn allows you to scale your WAS by launching N worker processes.

If you use "async-await" clause, then the FastAPI will only allocate 1 worker thread per process, and will use event loop by using uvloop. So rather than having N threads to run multiple jobs, register the job to the event loop and wait until finish, and only process the finished job by using the 1 worker thread. That is how the FastAPI works in async manner.

And to scale it horizontally within same machine, you could just use gunicorn with 2 or more workers, which will launch multiple processes (and the gunicorn will manage those processes).

Answer 2

The issues with Java threads in the question are addressed by the project Loom, which is now included in Jdk21. It is very well explained here https://www.baeldung.com/openjdk-project-loom :

Presently, Java relies on OS implementations for both the continuation [of threads] and the scheduler [for threads].

Now, in order to suspend a continuation, it's required to store the entire call-stack. And similarly, retrieve the call-stack on resumption. Since the OS implementation of continuations includes the native call stack along with Java's call stack, it results in a heavy footprint.

A bigger problem, though, is the use of OS scheduler. Since the scheduler runs in kernel mode, there's no differentiation between threads. And it treats every CPU request in the same manner. (...) For example, consider an application thread which performs some action on the requests and then passes on the data to another thread for further processing. Here, it would be better to schedule both these threads on the same CPU. But since the [OS] scheduler is agnostic to the thread requesting the CPU, this is impossible to guarantee.

The question really boils down to Why are OS threads considered expensive?

Answer 3

It very much depends on what your threads are doing. Imagine that your request is calling some external endpoint. Most of the time your java thread (which is heavy) is just doing nothing. Is it a problem? Depends. In java ecosystem threads are not created on the fly, that problem is solved in any decent framework with threadpools.

But still thread stack size occupies memory. If you would have 1000 threads (or 1000 concurent requests) in a thread pool waiting for IO operation your memory might go beyond 1GB easily.

For such particular scenario some async / await abstraction could utilise memory much better. Whether all task would be executed faster is a different story and depends on many factors.

So truly you need to go deeper, what scaling you are talking about? memory? or speed of execution of all operations (which usually also involve some cpu).

Not neglectable pros of java is that requests handled by threads can run in parallel and utilize many processors while in python this is simply not possible.

FastAPI and their async await model is easy to use, cuts developer off from threading model and from thinking about concurrency. Unfortunately it also means smaller control in case this is needed.

Just calling async method in python is not making any code execution faster. In the end under the hood there is one event loop that runs on one Cpu only.

Depending on the usecase it's good enough and efficient or just the opposite.

Usually people from Python ecosystem are hyped against heavy java and just use argument that it's faster / lighter without seeing other cons.

Here also a link to tease you a bit with typical webapp performance comparison across few frameworks (in web / rest scenarios)

https://www.travisluong.com/fastapi-vs-fastify-vs-spring-boot-vs-gin-benchmark/

Answer 4

FastAPI is a fast framework, and you can quickly (and easily) create API backends in it. To be honest, if you are a Java developer, I would recommend Quarkus or something for building a REST API, not FastAPI. FastAPI is a fantastic tool, absolutely great if you are already in the Python ecosystem.

When it goes about multithreading; Java is 'real' multithreading where as Python is very much not. Java threads will run concurrently; two tasks can and will be executed at the same time. In Python, within one Python process, this is (nearly) impossible. The reason for this is GIL (google it, there is ton's of stuff out there on how it works). The result is; even if you use 'real' threads in Python, code is still not executed concurrently but rather serially, where the interpreter (big difference to Java) is jumping from one call stack to another constantly.

As to what you refer to as 'logical threads', I think you mean the asynchronous capability of Python. This is basically the same as using threads (not really, but on an abstract level they are very similar); tasks are not run concurrently. There is just one thread that constantly switches between tasks. Tasks will yield back control to the event loop (the object that coordinates tasks and decides what is executed in which order), and another task is further executed until that task yields control, etc. It is basically the same kind of execution pattern as with threads within Python.

Comparing a Python framework to a Java framework is just weird in my opinion. They are both useful and cool, but not really competitors.