Reputation: 1191
When performing long downloads with PlayWS, which uses CompletableFuture
, these sometimes reach the defined request timeout. When that happens, then PlayWS doesn't seem to throw an exception (at least in my configuration), so the download can't be marked as failed and is processed although the data is corrupt.
Please excuse this abomination of code:
final CompletionStage<WSResponse> futureResponse = this.playWS.client
.url(importSource.getDownloadUrl())
.setMethod(HttpMethod.GET)
.setRequestTimeout(Duration.ofSeconds(5)) // When the timeout is reached, the download gets canceled
.stream();
try {
futureResponse
.thenAccept(res -> {
try (OutputStream outputStream = Files.newOutputStream(file.toPath())) {
final Source<ByteString, ?> responseBody = res.getBodyAsSource();
final Sink<ByteString, CompletionStage<Done>> outputWriter =
Sink.foreach(bytes -> {
outputStream.write(bytes.toArray());
});
responseBody
.runWith(outputWriter, this.playWS.materializer)
.whenComplete((value, error) -> {
System.out.println("VALUE: "+value); // == "Done"
System.out.println("Error: "+error); // == null
})
.exceptionally(exception -> {
throw new IllegalStateException("Download failed for: " + importSource.getDownloadUrl(), exception);
})
.toCompletableFuture().join();
} catch (final IOException e) {
throw new IllegalStateException("Couldn't open or write to OutputStream.", e);
}
})
.exceptionally(exception -> {
throw new IllegalStateException("Download failed for: " + importSource.getDownloadUrl(), exception);
})
.toCompletableFuture().get();
} catch (InterruptedException | ExecutionException e) {
throw new IllegalStateException("Couldn't complete CompletableFuture.", e);
}
Am I doing something fundamentally wrong or is this a bug?
The only solutions I see are:
Content-Length header
.Thanks for any suggestion.
Upvotes: 0
Views: 173
Reputation: 992
I think there is some over-complication.
You can just attach from the Future
or CompletableStage
to Source
. Akka Streams has a more robust API than CompletableFuture
(my opinion)
final CompletionStage<WSResponse> futureResponse = this.playWS.client
.url(importSource.getDownloadUrl())
.setMethod(HttpMethod.GET)
.setRequestTimeout(Duration.ofSeconds(5))
.stream();
Source<WSResponse> source = Source.fromCompletableStage(futureResponse);
source.map(...).filter(...).recover(...).runforeach(..., playWS.materializer)
Upvotes: 1