CODEWITHSUNDEEP
scalaplayframeworkplayframework-2.2asynchttpclient
jilen
jilen

Reputation: 5763

play ws perform two (or more) gzip requests concurrently sometimes fail

MyRequest:

WS.url(url)
.withHeaders("Accept-Encoding" -> "gzip")
.withQueryString("xxx","xxx")

Code used to decompress the body

 def call[T](api: WeiboApi[T])(implicit mf: Manifest[T]) = {
    val param = parameters(api)
    (api match {
      case _: Get[T] => get(api.url, param)
      case _: Post[T] => post(api.url, param)
    }) map {
      resp =>
      try {
        val decompressedBody = decompressIfGzip(resp)
        api.parse(decompressedBody)
      } catch {
        case e: WeiboApiError => throw e
        case e: Exception =>
          throw new Exception("cannot parse body api " + api, e)
      }
    }
  }

private def decompressIfGzip(resp: Response) = {
    val ahcResp = resp.getAHCResponse
        ahcResp.getHeader("Content-Encoding") match {
          case "gzip" | "GZIP" =>
            val in = ahcResp.getResponseBodyAsStream
            val gzipStream = new GZIPInputStream(in)
            try {
              val source = scala.io.Source.fromInputStream(gzipStream)
              source.mkString
            } finally {
              in.close()
            }
          case _ =>
            ahcResp.getResponseBody
        }
}

Then I send two request at the same time.

Sometimes it is fine

But sometimes it reprots

[error] Corrupt GZIP trailer
[error] sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
[error] sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
[error] sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
[error] lib.weibo.Weibo$.lib$weibo$Weibo$$decompressIfGzip(Weibo.scala:138)
[error] lib.weibo.Weibo$$anonfun$call$1.apply(Weibo.scala:47)
[error] lib.weibo.Weibo$$anonfun$call$1.apply(Weibo.scala:42)
[error]

akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) [error] akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) [error] akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) [error] akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) [error] akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) [error] akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) [error] akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

What's wrong ?

Upvotes: 1

Views: 359

Answers (2)

jilen
jilen

Reputation: 5763

Turns out the response data sometimes is corrupt.

Upvotes: 1

Will Sargent
Will Sargent

Reputation: 4396

Not sure, but you may need to wrap that code with an InputStreamReader:

https://stackoverflow.com/questions/3627401/gzipinputstream-to-string/3627441#3627441

Upvotes: 0

Related Questions