CODEWITHSUNDEEP
spring-mvcasynchronousspring-webfluxproject-reactorspring-webclient
Asif Ansari
Asif Ansari

Reputation: 93

Making pagination API calls simulaneously using Spring Webclient

I have to make paginated API calls, using Webclient and combine all the result at last. For ex : Person latest 1000 transaction details. In one call, i am gonna get max 100 Objects in json response (List). I can only get maximum 1000 records for this Person.

In pseudo code - java, it could look something like this

    int Total = 1000;
    int maxEachCall = 100;
    int noOfCalls = maxTotal/maxEachCall;
    int startIndex = 0;

    List<MyDto> mailList = new ArrayList();

    for(int i =startIndex; i<noOfCalls ;i+100){
        List<MyDto> list =  webClient.get().uri(baseUri+"?startIndex"+startIndex)
                        .retrieve()
                        .bodyToMono(MyDto.class)                            
                        .retry(3)
                        .block();
        mainList.addAll(list);
        if(startIndex>900 || list.size()<100){
            break;
        }
    }

How can i write the same thing in reactive way WITHOUT blocking in spring mvc ?

Something like this??? I dont know. Help me

    int Total = 1000;
    int maxEachCall = 100;
    int noOfCalls = maxTotal/maxEachCall;
      int startIndex = 0;
    Flux.range(1,noOfCalls)
        .flatMap(
            // tried to call api using startIndex but can't use startIndex here or manipulate it
        )
        .doOnError(e->LOGGER.info("Exception occured ",e))
        .blockLast(); ???

Upvotes: 1

Views: 2124

Answers (1)

Michael Berry
Michael Berry

Reputation: 72254

You could do something similar to:

Flux.range(0, 10)
        .map(start -> start * 100)
        .flatMap(
                start -> webClient.get()
                        .uri(baseUri + "?startIndex=" + start)
                        .retrieve()
                        .bodyToMono(MyDtoCollection.class)
                        .map(c -> splitCollectionIntoIndividualMyDto(c)) //or something to that effect, since you'll get 100 `MyDto` at a time and presumably need to split them.
        )
        .collectList();

...which will provide you with a Mono<List<MyDto>> containing all your paginated results. If your stack is fully reactive you would then continue to use that Mono in the reactive chain elsewhere. If not (yet) fully reactive, and you're just using reactor for this bit, you can then choose to block() at the most sensible location.

Upvotes: 1

Related Questions