CODEWITHSUNDEEP
javamultithreadinghttpasynchronousjmeter
Jade Tang
Jade Tang

Reputation: 319

Asynchronous and synchronous HTTP request on server side, performance comparison

I am trying to figure out the pros and cons of asynchronous and synchronous HTTP request processing. I am using the Dropwizard with Jersey as my framework. The test is comparing the asynchronous and synchronous HTTP request processing, this is my code

@Path("/")
public class RootResource {

    ExecutorService executor;

    public RootResource(int threadPoolSize){
        executor = Executors.newFixedThreadPool(threadPoolSize);
    }

    @GET
    @Path("/sync")
    public String sayHello() throws InterruptedException {
        TimeUnit.SECONDS.sleep(1L);
        return "ok";
    }

    @GET
    @Path("/async")
    public void sayHelloAsync(@Suspended final AsyncResponse asyncResponse) throws Exception {
        executor.submit(() -> {
            try {
                doSomeBusiness();
                asyncResponse.resume("ok");
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        });
    }


    private void doSomeBusiness() throws InterruptedException {
        TimeUnit.SECONDS.sleep(1L);
    }

}

The sync API will run in the worker thread maintained by the Jetty and the async API will mainly run in the customs thread pool. And here is my result by Jmeter

My question would be: What's the differences between these two approaches, and which pattern should I use in which scenario?

Related topic : Performance difference between Synchronous HTTP Handler and Asynchronous HTTP Handler

update

I run the test with 10 delays as suggested

sync-500-server-thread

async-500-workerthread

Upvotes: 13

Views: 5614

Answers (5)

JojOatXGME
JojOatXGME

Reputation: 3296

The main performance advantage of asynchronous programming is that you can reduce the amount of threads on your system. The documentation of Jetty looks like a good reference.

The servlet API (pre 2.5) supports only a synchronous call style, so that any waiting that a servlet needs to do must be with blocking. Unfortunately this means that the thread allocated to the request must be held during that wait along with all its resources: kernel thread, stack memory and often pooled buffers, character converters, EE authentication context, etc. It is wasteful of system resources to hold these resources while waiting. Significantly better scalability and quality of service can be achieved if waiting is done asynchronously.

The code of the question does not tap into this advantage because it still blocks a thread for each request, it is just a different thread from a custom thread pool:

@GET
@Path("/async")
public void sayHelloAsync(@Suspended final AsyncResponse asyncResponse) throws Exception {
    executor.submit(() -> {
        try {
            doSomeBusiness();
            asyncResponse.resume("ok");
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    });
}


private void doSomeBusiness() throws InterruptedException {
    TimeUnit.SECONDS.sleep(1L);
}

Note that this implementation is especially problematic because in addition to blocking a thread, the amount of threads is strictly limited by the thread pool.

If you want to tap into the advantages of asynchronous request handling, you have to reimplement TimeUnit.SECONDS.sleep(1L), so that it does not block any thread.

Upvotes: 0

RAJKUMAR NAGARETHINAM
RAJKUMAR NAGARETHINAM

Reputation: 1538

The following are my thoughts.

Whether its synchronous or asynchronous request, its nothing related to the performance of HTTP but it related to your application's performance

Synchronous requests will block the application until it receives the response, whereas in asynchronous request basically, you will assign this work in a separate worker thread which will take care of the rest of things. So in asynchronous design, your main thread still can continue its own work.

Let say due to some limitation(not resource limitation of the server) your server can handle a limited number of connections (Basically each and every connection will be handled in a separate thread differs between the server we use). If your server can handle more number of threads than the connections and also if you don't want to return any data as a result of async work you have created, then you can design asynchronous logic. Because you will create a new thread to handle the requested task.

But if you expect results of the operations to be returned in the response nothing will differ.

Upvotes: 8

Ori Marko
Ori Marko

Reputation: 58882

You are using @Suspended combined with async which still wait for response

@Suspended will pause/Suspend the current thread until it gets response

If you want to get better performance in async, write a different async method with immediate response using ExecutorService and Future

private ExecutorService executor;
private Future<String> futureResult;
@PostConstruct
public void onCreate() {
    this.executor = Executors.newSingleThreadExecutor();
}
@POST
public Response startTask() {
    futureResult = executor.submit(new ExpensiveTask());
    return Response.status(Status.ACCEPTED).build();
}
@GET
public Response getResult() throws ExecutionException, InterruptedException {
    if (futureResult != null && futureResult.isDone()) {
        return Response.status(Status.OK).entity(futureResult.get()).build();
    } else {
        return Response.status(Status.FORBIDDEN).entity("Try later").build();
    }
}

Upvotes: 5

BogdanSucaciu
BogdanSucaciu

Reputation: 904

Let's consider the following scenario:

Single Backend system
                    ____________
                   |  System A  |
 HTTP Request -->  |            |
                   |  1.        |
                   |  2.        |
 HTTP Response <-- |            |
                   |____________|

You have one backend system which does some processing based on the request received on a particular order ( operation 1 and then operation 2 ). If you process the request synchronously or asynchronously doesn't really matter, it's the same amount of computation that needs to be done ( maybe some slight variations like you have encountered in your test ).

Now, let's consider a multi-backend scenario:

Multi-Backend System
                        ____________
                       |  System A  |       __________
     HTTP Request -->  |            | -->  |          |
                       |  1.        |      | System B |
                       |            | <--  |__________|
                       |            |       __________  
                       |  2.        | -->  |          |
     HTTP Response <-- |            |      | System C |
                       |____________| <--  |__________|

Still, 2 processing steps required to be done but this time, on each step we will call another back'end system.

SYNC processing:

  1. Call System B
  2. Wait for a response from System B
  3. Call System C
  4. Wait for a response from System C

Total time spent: B + C

ASYNC processing:

  1. Call System B
  2. Go forward since the call is not blocking
  3. Call System C
  4. Go forward since the call is not blocking
  5. Receive a response from System B
  6. Receive a response from System C
  7. Complete the call to the client

Total time spent: max(B, C)

Why max? Since all the calls are non-blocking then you will have to wait just for the slowest back'end to reply.

Upvotes: 3

shazin
shazin

Reputation: 21923

I am writing this from my personal experience writing a Promotions engine using Asynchronous handlers for Sri Lanka's front running Taxi Hailing service, which even competes shoulder to shoulder with Uber.

It is more scale-ability, availability and resource utilization than performance when it comes to using Asynchronous handlers over synchronous handlers.

If you are using synchronous handlers, your maximum concurrent no of requests is defined by the no of threads available to accept new connections after which your service can no longer accept request at all.

Where as if you use asynchronous handlers, the accept thread count has nothing to do with no of concurrent requests you can cater. Thus your service can scale from 100 rps to 1 million rps and can have high availability.

If you are concerned about latency and throughput, you can get good improvements if you are using Non Blocking APIs with Asynchronous Handlers. Non Blocking Socket (NIO), Non Blocking Storage (Mongo DB Reactive, Redis Reactive), Message Queues (Kafka, RabbitMQ) etc.

Upvotes: 0

Related Questions