How do I get a byte[] (image) from a webservice using micronaunt HttpClient

Question

I am porting a Grails 3.1 library for using some internal webservices to Grails 4.0. One of the services provides an image of a requested employee upon request. I am having difficulty implementing the (micronaut) HttpClient code to process the request - specifically to get a proper byte[] that is the returned image.

A simple curl command on the command line works with the service:

curl -D headers.txt -H 'Authorization:Basic <encodedKeyHere>' https:<serviceUrl> >> image.jpg

and the image is correct. The header.txt is:

HTTP/1.1 200 
content-type: image/jpeg;charset=UTF-8
date: Tue, 27 Aug 2019 20:05:43 GMT
x-ratelimit-limit: 100
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 99
x-ratelimit-remaining: 99
X-RateLimit-Reset: 38089
x-ratelimit-reset: 15719
Content-Length: 11918
Connection: keep-alive

The old library uses the groovyx.net.http.HTTPBuilder and simply does:

http.request(Method.GET, ContentType.BINARY) {
            uri.path = photoUrlPath
            uri.query = queryString
            headers.'Authorization' = "Basic $encoded".toString()
            response.success = { resp, inputstream ->
                log.info "response status: ${resp.statusLine}"                
                return ['status':resp.status, 'body':inputstream.getBytes()]
            }
            response.failure = { resp ->
                return ['status':resp.status, 
                        'error':resp.statusLine.reasonPhrase, 
                         body:resp.getEntity().getContent().getText()]
            }
        }

so returning the bytes from an inputStream. This works.

I've tried several things using the micronaut HttpClient, both with the low level API and with the declarative API.

A simple example with the declarative API:

    @Get(value='${photo.ws.pathurl}', produces = MediaType.IMAGE_JPEG)
    HttpResponse<byte[]> getPhoto(@Header ('Authorization') String authValue, 
                                  @QueryValue("emplId") String emplId)

And than in the Service:

    HttpResponse<byte[]> resp = photoClient.getPhoto(getBasicAuth(),emplId)
    def status = resp.status()            // code == 200 --> worked 
    def bodyStrOne = resp.getBody()       // nope: get Optional.empty 
    // Tried different getBody(class) -> Can't figure out where the byte[]s are   
    // For example can do:
    def buf = resp.getBody(io.netty.buffer.ByteBuf).value // Why need .value?
    def bytes = buf.readableBytes()       // Returns 11918 --> the expected value
    byte[] ans = new byte[buf.readableBytes()]
    buf.readBytes(ans)    // Throws exception: io.netty.util.IllegalReferenceCountException: refCnt: 0

This "works" but the returned String looses some encoding that I can't reverse:

   // Client - use HttpResponse<String>
   @Get(value='${photo.ws.pathurl}', produces = MediaType.IMAGE_JPEG)
   HttpResponse<String> getPhoto(@Header ('Authorization') String authValue, 
                                  @QueryValue("emplId") String emplId)

   // Service 
   HttpResponse<String> respOne = photoClient.getPhoto(getBasicAuth(),emplId)
   def status = respOne.status()                  // code == 200 --> worked 
   def bodyStrOne = respOne.getBody(String.class) // <-- RETURNS DATA..just NOT an Image..or encoded or something
   String str = bodyStrOne.value                  // get the String data  
   // But these bytes aren't correct
   byte[] ans = str.getBytes()                    // NOT an image..close but not.
   // str.getBytes(StandardCharsets.UTF_8) or any other charset doesn't work 

Everything I've tried with the ByteBuf classes throws the io.netty.util.IllegalReferenceCountException: refCnt: 0 exception.

Any direction/help would be greatly appreciated.

Running:

    Grails   4.0 
    JDK      1.8.0_221 
    Groovy   2.4.7
    Windows  10
    IntellJ  2019.2  

Answer 1

Documentation propose to send bytes via input stream but I didn't manage to make it work. The most brittle thing that HttpClient should Consume bytes but Server should Produce.

@Get(value = "/write", produces = MediaType.TEXT_PLAIN)
HttpResponse<byte[]> write() {
    byte[] bytes = "test".getBytes(StandardCharsets.UTF_8);
    return HttpResonse.ok(bytes); // 
}

Answer 2

We are currently using this implementation:

@Client(value = "\${image-endpoint}")
interface ImageClient {

  @Get("/img")
  fun getImageForAddress(
    @QueryValue("a") a: String
  ): CompletableFuture<ByteArray>
}

works fine for us.

When I use the HttpResponse I get an error as well, couldn't make it work with that.

Answer 3

It must be Grails bug.

---

Add this line into logback.groovy:

logger("io.micronaut.http", TRACE)

Then you should see that the body was not empty but finally it ends with error Unable to convert response body to target type class [B. See the trace:

2019-09-11 11:19:16.235 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : Status Code: 200 OK
2019-09-11 11:19:16.235 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : Content-Type: image/jpeg
2019-09-11 11:19:16.235 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : Content-Length: 11112
2019-09-11 11:19:16.237 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : Accept-Ranges: bytes
2019-09-11 11:19:16.237 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : Response Body
2019-09-11 11:19:16.237 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : ----
2019-09-11 11:19:16.238 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : ���� C   
...
2019-09-11 11:19:16.241 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : ----
2019-09-11 11:19:16.243 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient        : Unable to convert response body to target type class [B

But when you try the same in standalone Microunaut application (add <logger name="io.micronaut.http" level="trace"/> into logback.xml) the result is different:

09:02:48.583 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - Status Code: 200 OK
09:02:48.583 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - Content-Type: image/jpeg
09:02:48.589 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - content-length: 23195
09:02:48.590 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - Response Body
09:02:48.590 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - ----
09:02:48.612 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - ���� C���
...
09:02:48.620 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - ----

Micronaut trace has no error.

---

Here is an example of declarative HTTP client which downloads random image from https://picsum.photos web site:

import io.micronaut.http.HttpResponse
import io.micronaut.http.MediaType
import io.micronaut.http.annotation.Get
import io.micronaut.http.client.annotation.Client

@Client('https://picsum.photos')
interface LoremPicsumClient {
    @Get(value = '{width}/{height}', produces = MediaType.IMAGE_JPEG)
    HttpResponse<byte[]> getImage(Integer width, Integer height)
}

And Spock unit test for it:

import io.micronaut.http.HttpStatus
import io.micronaut.test.annotation.MicronautTest
import spock.lang.Specification

import javax.inject.Inject
import java.nio.file.Files
import java.nio.file.Paths

@MicronautTest
class LoremPicsumClientSpec extends Specification {
    @Inject
    LoremPicsumClient client

    void 'image is downloaded'() {
        given:
        def output = Paths.get('test')

        when:
        def response = client.getImage(300, 200)

        then:
        response.status == HttpStatus.OK
        response.getBody().isPresent()

        when:
        Files.write(output, response.getBody().get())

        then:
        Files.probeContentType(output) == 'image/jpeg'
    }
}

In Micronaut the test passes and an image is saved into the test file. But in Grails the test fails because HttpClient is not able to convert the response bytes into byte array or better into anything else then String.