CODEWITHSUNDEEP
javamultithreadingrestasynchronousmicroservices
tattihead
tattihead

Reputation: 81

Async HTTP request vs HTTP requests on new thread

I have 2 microservices (A and B).

A has an endpoint which accepts POST requests. When users make a POST request, this happens:

  1. Service A takes the object from the POST request body and stores it in a database.
  2. Service A converts the object to a different object. And the new object gets sent to service B via Jersey HTTP client.

Step 2 takes place on a Java thread pool I have created (Executors.newCachedThreadPool). By doing step 2 on a new thread, the response time of service A's endpoint is not affected.

However, if service B is taking long to respond, service A can potentially create too many threads when it is receiving many POST requests. To help fix this, I can use a fixed thread pool (Exectuors.newFixedThreadPool).

In addition to the fixed thread pool, should I also use an asynchronous non-blocking HTTP client? Such as the one here: https://hc.apache.org/httpcomponents-asyncclient-dev/. The Jersey HTTP client that I use is blocking.

It seems like it is right to use the async HTTP client. But if I switch to a fixed thread pool, I think the async HTTP client won't provide a significant benefit - am I wrong in thinking this?

Upvotes: 2

Views: 3819

Answers (3)

TwoThe
TwoThe

Reputation: 14269

The reason to use a non-blocking HTTP-Client is to prevent too much CPU from being used on thread-switching. If you already solve that problem by limiting the amount of background threads, then non-blocking IO won't provide any noticeable benefits.

There is another problem with your setup: it is very vulnerable to DDOS attacks (intentional or accidental ones). If a someone calls your service very often, it will internally create a huge work-load that will keep the service busy for a long time. You will definitely need to limit the background task queue (which is a supported feature of the Executor class) and return 503 (or equivalent) if there are too many pending tasks.

Upvotes: 0

albert_nil
albert_nil

Reputation: 1658

The difference is that with sync client, step A thread will be doing a connection to step 2 endpoint and wait for a response. Making step 2 implementation async will and just return 200 directly (or whatever) will help on decreasing waiting time; but it will still be doing the connection and waiting for response.

With non-blocking client instead, the step A call itself will be done by another thread. So everything is untied from step A thread. Also, system can make use of that thread for other stuff until it gets a response from step B and needs to resume work.

The idea is that your origin threads will not be idle so much time waiting for responses, but instead being reused to do other work while in between.

Upvotes: 0

Danylo Zatorsky
Danylo Zatorsky

Reputation: 6104

Even if you use fixed thread pool all your threads in it will be blocked on step 2 meaning that they won't do any meaningful job - just wait for your API to return a response which is not a pragmatic resource management. In this case, you will be able to handle a limited amount of incoming requests since threads in the thread pool will be always busy instead of handling new requests.

In the case of a non-blocking client, you are blocking just one single thread (let's call it dispatcher thread) which is responsible for sending and waiting for all the requests/responses. It will be running in a "while loop" (you could call it an event loop) and check whether all the packages were received as a response so they are ready for worker threads to be picked up.

In the latter scenario, you get a larger amount of available threads ready to do some meaningful job, so your throughput will be increased.

Upvotes: 1

Related Questions