CODEWITHSUNDEEP
spring-bootspring-webfluxproject-reactor
Timothy
Timothy

Reputation: 1035

Spring reactive : mixing RestTemplate & WebClient

I have two endpoints : /parent and /child/{parentId}
I need to return list of all Child

public class Parent {
  private long id;
  private Child child;
}

public class Child {
  private long childId;
  private String someAttribute;
}

However, call to /child/{parentId} is quite slow, so Im trying to do this:

  1. Call /parent to get 100 parent data, using asynchronous RestTemplate
  2. For each parent data, call /child/{parentId} to get detail
  3. Add the result call to /child/{parentId} into resultList
  4. When 100 calls to /child/{parentId} is done, return resultList

I use wrapper class since most endpoints returns JSON in format :

{
  "next": "String",
  "data": [
    // parent goes here
  ]
}

So I wrap it in this

public class ResponseWrapper<T> {
  private List<T> data;
  private String next;
}

I wrote this code, but the resultList always return empty elements. What is the correct way to achieve this?

public List<Child> getAllParents() {
    var endpointParent = StringUtils.join(HOST, "/parent");
    var resultList = new ArrayList<Child>();

    var responseParent = restTemplate.exchange(endpointParent, HttpMethod.GET, httpEntity,
            new ParameterizedTypeReference<ResponseWrapper<Parent>>() {
            });

    responseParent.getBody().getData().stream().forEach(parent -> {
        var endpointChild = StringUtils.join(HOST, "/child/", parent.getId());
        // async call due to slow endpoint child
        webClient.get().uri(endpointChild).retrieve()
                .bodyToMono(new ParameterizedTypeReference<ResponseWrapper<Child>>() {
                }).map(wrapper -> wrapper.getData()).subscribe(children -> {
                    children.stream().forEach(child -> resultList.add(child));
                });
    });

    return resultList;
}

Upvotes: 1

Views: 2104

Answers (1)

Brian Clozel
Brian Clozel

Reputation: 59211

Calling subscribe on a reactive type starts the processing but returns immediately; you have no guarantee at that point that the processing is done. So by the time your snippet is calling return resultList, the WebClient is probably is still busy fetching things.

You're better off discarding the async resttemplate (which is now deprecated in favour of WebClient) and build a single pipeline like:

public List<Child> getAllParents() {
    var endpointParent = StringUtils.join(HOST, "/parent");
    var resultList = new ArrayList<Child>();

   Flux<Parent> parents = webClient.get().uri(endpointParent)
            .retrieve().bodyToMono(ResponseWrapper.class)
            .flatMapMany(wrapper -> Flux.fromIterable(wrapper.data));

    return parents.flatMap(parent -> {
      var endpointChild = StringUtils.join(HOST, "/child/", parent.getId());
      return webClient.get().uri(endpointChild).retrieve()
                .bodyToMono(new ParameterizedTypeReference<ResponseWrapper<Child>>() {
                }).flatMapMany(wrapper -> Flux.fromIterable(wrapper.getData()));
    }).collectList().block();
}

By default, the parents.flatMap operator will process elements with some concurrency (16 by default I believe). You can choose a different value by calling another variant of the Flux.flatMap operator with a chosen concurrency value.

Upvotes: 1

Related Questions