Best way to make multiple asynchronous calls to same web service

Question

Application on which I am working is going to consume 2 REST web service in below sequence:

1) Count Records - To know the numbers of records within a particular time frame.

2) Fetch Records - Once we have number of records then we need to call this service. But this service has a threshold to fetch 10K records in a go. Lets say if first service tell me within particular time interval, it has 100K of records, then I need to call second web service 10 times in paginated way considering it's threshold is 10K in one go.

So if I will make 10 synchronous calls, my application would be too slow to respond back. So I need to a mechanism to make asynchronous calls.

I am using spring framework in the back end code and using rest template for web service call. I am looking to find the best way to make asynchronous call to the above mentioned POST web service

I have done some research and found Asynchronous method useful as below: https://spring.io/guides/gs/async-method/

Can you please guide me if this is a right approach what I am looking at or is their a better way to make asynchronous call? Looking for your suggestions, Thanks!

Answer 1

This is just an improvement over @shawn answer. With the implementation provided earlier, sometimes I was facing issue due to below block:

while (responses < futures.size()) {        
   for (Future<ResponseEntity<List<Record>>> future : futures) {
       if (future.isDone()) {
            responses++;
            try {
                ResponseEntity<List<Record>> responseEntity = future.get();
                fullListOfRecords.addAll(responseEntity.getBody());
            } catch (InterruptedException | ExecutionException e) {
                e.printStackTrace();
        }
    }
}

Here if 6 threads are processed, sometimes same thread is allowed to enter multiple times in above block so it ends up having duplicate records in response. So to avoid such situation, I have added callback block to create final response which make sure no duplicate response, though we still need while(responses < futures.size()) loop block with future.get() to block method to return final combined response until all the asynchronous calls are processed.

@Component
public class SampleAsyncService {
private RestTemplate restTemplate;
private AsyncRestTemplate asyncRestTemplate;

@Value("${myapp.batchSize:1000}")
private int batchSize;

public SampleAsyncService(AsyncRestTemplate asyncRestTemplate, RestTemplate restTemplate) {
    this.asyncRestTemplate = asyncRestTemplate;
    this.restTemplate = restTemplate;
}

public List<Record> callForRecords() {
    ResponseEntity<Integer> response = restTemplate.getForEntity("http://localhost:8081/countService",
            Integer.class);
    int totalRecords = response.getBody().intValue();
    List<Future<ResponseEntity<List<Record>>>> futures = new ArrayList<Future<ResponseEntity<List<Record>>>>();
    for (int offset = 0; offset < totalRecords;) {
        ListenableFuture<ResponseEntity<List<Record>>> future = asyncRestTemplate.exchange(
                "http://localhost:8081/records?startRow={}&endRow={}", HttpMethod.GET, null,
                new ParameterizedTypeReference<List<Record>>() {
                }, offset, batchSize);
        future.addCallback(
                new ListenableFutureCallback<ResponseEntity<ChatTranscript>>() {  
                    @Override  
                    public void onSuccess(ResponseEntity<ChatTranscript> response) {  
                        fullListOfRecords.addAll(responseEntity.getBody());
                        log.debug("Success: " + Thread.currentThread());  
                    }  

                    @Override  
                    public void onFailure(Throwable t) {
                        log.debug("Error: " + Thread.currentThread());
                    }  
                }
            );
        futures.add(future);
        offset = offset + batchSize;
    }

    int responses = 0;
    List<Record> fullListOfRecords = new ArrayList<Record>();
    while (responses < futures.size()) {
        for (Future<ResponseEntity<List<Record>>> future : futures) {
            if (future.isDone()) {
                responses++;
                try {
                    future.get();
                } catch (InterruptedException | ExecutionException e) {
                    e.printStackTrace();
                }
            }
        }
    }

    return fullListOfRecords;
}

public class Record {
}

}

Answer 2

What @Journycorner linked is a good start but it doesn't really show the whole picture as it only makes a single request. Working with Future is definitely on the right path. The fact that Spring 4 offers an AsyncRestTemplate that returns a Future is exactly what you want to use.

On my phone so can't write up the full code but this is roughly what you want to do.

@Component
public class SampleAsyncService {
    private RestTemplate restTemplate;
    private AsyncRestTemplate asyncRestTemplate;

    @Value("${myapp.batchSize:1000}")
    private int batchSize;

    public SampleAsyncService(AsyncRestTemplate asyncRestTemplate, RestTemplate restTemplate) {
        this.asyncRestTemplate = asyncRestTemplate;
        this.restTemplate = restTemplate;
    }

    public List<Record> callForRecords() {
        ResponseEntity<Integer> response = restTemplate.getForEntity("http://localhost:8081/countService",
                Integer.class);
        int totalRecords = response.getBody().intValue();
        List<Future<ResponseEntity<List<Record>>>> futures = new ArrayList<Future<ResponseEntity<List<Record>>>>();
        for (int offset = 0; offset < totalRecords;) {
            ListenableFuture<ResponseEntity<List<Record>>> future = asyncRestTemplate.exchange(
                    "http://localhost:8081/records?startRow={}&endRow={}", HttpMethod.GET, null,
                    new ParameterizedTypeReference<List<Record>>() {
                    }, offset, batchSize);
            futures.add(future);
            offset = offset + batchSize;
        }

        int responses = 0;
        List<Record> fullListOfRecords = new ArrayList<Record>();
        while (responses < futures.size()) {
            for (Future<ResponseEntity<List<Record>>> future : futures) {
                if (future.isDone()) {
                    responses++;
                    try {
                        ResponseEntity<List<Record>> responseEntity = future.get();
                        fullListOfRecords.addAll(responseEntity.getBody());
                    } catch (InterruptedException | ExecutionException e) {
                        e.printStackTrace();
                    }
                }
            }
        }

        return fullListOfRecords;
    }

    public class Record {
    }
}

* Update * Created complete code sample.

Answer 3

I am assuming you need all the 100k records in a single execution so as to package all the data maybe into a file or perform some business logic in one go . If thats not case it would be wise to reconsider the need to load all data into single execution straining the memory usage of jvm or running into Outofmemory errors.

Assuming the former, Async could be an option to execute parallel threads and capture and collate responses from each. However you need to keep an upper limit of number of threads to be executed in parallel using a "thread pool size" of a task executor. One more reason to limit the thread size is avoid loading your partner rest webservice with too many parallel calls. Eventually the partner webservice is loading data from database, which will give optimum performance for certain limit of parallel query executions. Hope this helps!

Answer 4

If it really has to be asynchronous using the Spring Implementation seems like a smart idea.