CODEWITHSUNDEEP
scalaspray
vptheron
vptheron

Reputation: 7466

Use Ack on chunked responses with Spray

I'm using spray-can 1.2.1.

I'm streaming big files from/to a storage, I use both chunked requests and chunked responses for that.

For chunk requests I use the built-in ack mechanism in my actor to make sure each chunk has been written before sending more:

connection ! MessageChunk(data).withAck(ChunkSent)

connection is the IO actor provided by Spray and Akka, then I can wait for a ChunkSent before sending the next chunk. Good.

I'm struggling to reproduce the same behavior with chunked responses. I can send my HttpRequest and then receive a ChunkedResponseStart, followed by a bunch of MessageChunks and finally a ChunkedMessageEnd but is there a way to force Spray to wait for me to send an ack after each MessageChunk before sending the next one?

Edit: Just to be a bit more clear: I use spray-can as a client in this case, I am not the server, the server is the storage I mentioned before.

Upvotes: 4

Views: 1071

Answers (1)

jrudolph
jrudolph

Reputation: 8367

Well put question. Currently, you cannot make spray (1.x.1) wait for Acks before continuing to read.

What you can do however is to send Tcp.SuspendReading and Tcp.ResumeReading commands to the client connection (sender of chunks) to instruct the Akka IO TCP layer to stop reading while you are overloaded. See this proof-of-concept which tries to add Acking for the receive side (for the server but should work similarly for the client side) on top of SuspendReading/ResumeReading for hints about how to build something with the current version of spray.

This situation is clearly not optimal because especially under load 1) you need to figure out that you are overloaded and 2) those message may be stuck in the message queue of the TCP connection before they will be handled.

There are two things that will improve the situation in the future:

  • Since recently Akka IO supports a "pull mode" where the TCP connection will only ever read once and then wait for a Tcp.ResumeReading command (basically an Ack). However, this is not available for use in spray (and probably won't be).
  • What we are focusing on right now for Akka HTTP is streaming support. The plan is to introduce a new API (next to what we have in spray-can) that makes working with streams natural for HTTP and which will support automatic back-pressure support without any need for extra user code to support it. Alas, it is not yet ready.

Upvotes: 3

Related Questions