How to bind text response to json and put in object on a declarative client on micronaut?

Question

I made my declarative http client on my app built in micronaut. This need to consume a services which responds with text/html type.

I manage to get a list but with LinkedHashMap inside. And I want them to be objects of Pharmacy

My question is: how I can transform that response into a List of object?

@Client("${services.url}")
public interface PharmacyClient {
    @Get("${services.path}?${services.param}=${services.value}")
    Flowable<List<Pharmacy>> retrieve();
}

public class StoreService {

    private final PharmacyClient pharmacyClient;

    public StoreService(PharmacyClient pharmacyClient) {
        this.pharmacyClient = pharmacyClient;
    }

    public Flowable<List<Store>> all() {
        Flowable<List<Pharmacy>> listFlowable = this.pharmacyClient.retrieve();
        return listFlowable
                .doOnError(throwable -> log.error(throwable.getLocalizedMessage()))
                .flatMap(pharmacies ->
                        Flowable.just(pharmacies.stream() // here is a list of LinkedHashMap and i'd like to user Pharmacy objects
                                .map(pharmacy -> Store.builder().borough(pharmacy.getBoroughFk()).build())
                                .collect(Collectors.toList())
                        )
                );
    }
}

Code: https://github.com/j1cs/drugstore-demo/tree/master/backend

Answer 1

There is no fully-fledged framework AFAIK that provides support for HTML content to POJO mapping (which is usually referred to as scraping) as is the case for Micronaut, .

Meanwhile you can easily plug a converter bean based on jspoon intercepting and transforming your API results in equivalent POJOs:

class Root {
    @Selector(value = ".pharmacy") List<Pharmacy> pharmacies;
}

class Pharmacy {
    @Selector(value = "span:nth-child(1)") String name;
}

@Client("${services.minsal.url}")
public interface PharmacyClient {
    @Get("${services.minsal.path}?${services.minsal.param}=${services.minsal.value}")
    Flowable<String> retrieve();
}

@Singleton
public class ConverterService {

    public List<Pharmacy> toPharmacies(String htmlContent) {
        Jspoon jspoon = Jspoon.create();
        HtmlAdapter<Root> htmlAdapter = jspoon.adapter(Root.class);
        return htmlAdapter.fromHtml(htmlContent).pharmacies;
    }
}

public class StoreService {

    private final PharmacyClient pharmacyClient;
    private final ConverterService converterService;

    public StoreService(PharmacyClient pharmacyClient, ConverterService converterService) {
        this.pharmacyClient = pharmacyClient;
        this.converterService = converterService;
    }

    public Flowable<List<Store>> all() {
        Flowable<List<Pharmacy>> listFlowable = this.pharmacyClient.retrieve().map(this.converterService::toPharmacies)
        return listFlowable
                .doOnError(throwable -> log.error(throwable.getLocalizedMessage()))
                .flatMap(pharmacies ->
                        Flowable.just(pharmacies.stream() // here is a list of LinkedHashMap and i'd like to user Pharmacy objects
                                .map(pharmacy -> Store.builder().borough(pharmacy.getBoroughFk()).build())
                                .collect(Collectors.toList())
                        )
                );
    }
}

Answer 2

I ended up with this.

@Client("${services.url}")
public interface PharmacyClient {
    @Get(value = "${services.path}?${services.param}=${services.value}")
    Flowable<Pharmacy[]> retrieve();
}

public class StoreService {

    private final PharmacyClient pharmacyClient;

    public StoreService(PharmacyClient pharmacyClient) {
        this.pharmacyClient = pharmacyClient;
    }

    public Flowable<List<Store>> all() {
        Flowable<Pharmacy[]> flowable = this.pharmacyClient.retrieve();
        return flowable
                .switchMap(pharmacies ->
                        Flowable.just(Arrays.stream(pharmacies)
                                .map(pharmacyStoreMapping)
                                .collect(Collectors.toList())
                        )
                ).doOnError(throwable -> log.error(throwable.getLocalizedMessage()));
    }

}

Still I want to know if i can change arrays to list in the declarative client.
Meanwhile i think this it's a good option.

UPDATE

I have been wrong all this time. First of all I don't need to add a list to the flowable because when the framework exposes the service it responds with a list of elements already. So finally I did this:

@Client("${services.url}")
public interface PharmacyClient {
    @Get(value = "${services.path}?${services.param}=${services.value}")
    Flowable<Pharmacy> retrieve();
}

public class StoreService {

    private final PharmacyClient pharmacyClient;

    public StoreService(PharmacyClient pharmacyClient) {
        this.pharmacyClient = pharmacyClient;
    }

    public Flowable<Store> all() {
        Flowable<Pharmacy> flowable = this.pharmacyClient.retrieve();
        return flowable
                .switchMap(pharmacyPublisherFunction)
                .doOnError(throwable -> log.error(throwable.getLocalizedMessage()));

}

As we can see the http client automatically transform the text/html data into json and it parses it well. I don't know why really. Maybe @JeffScottBrown can gives us some hints.