CODEWITHSUNDEEP
scalaakkaspray
sapito
sapito

Reputation: 1522

Sending large files with Spray

I know very similar questions have been asked before. But I don't think the solutions I found on google/stackoverflow are suitable for me.

I started to write some web services with Scala/Spray, and it seems the best way to send large files without consuming large amouns of memory is using the stream marshalling. This way Spray will send http chunks. Two questions:

  1. Is it possible to send the file without using HTTP chunks and without reading the entire file into memory?

  2. AFAIK akka.io only process one write at a time, meaning it can buffer one write until it has been passed on to the O/S kernel in full. Would it be possible to tell Spray, for each HTTP response, the length of the content? Thereafter Spray would ask for new data (through akka messages) untill the entire content length is completed. Eg, I indicate my content length is 100 bytes. Spray sends a message asking for data to my actor, I provide 50 bytes. Once this data is passed on to the O/S, spray sends another message asking for new data. I provide the remaining 50 bytes... the response is completed then.

Upvotes: 4

Views: 1477

Answers (1)

jrudolph
jrudolph

Reputation: 8367

Is it possible to send the file without using HTTP chunks [on the wire]

Yes, you need to enable chunkless streaming. See http://spray.io/documentation/1.2.4/spray-routing/advanced-topics/response-streaming/

Chunkless streaming works regardless whether you use the Stream marshaller or provide the response as MessageChunks yourself. See the below example.

without reading the entire file into memory

Yes, that should work if you supply data as a Stream[Array[Byte]] or Stream[ByteString].

[...] Thereafter Spray would ask for new data [...]

That's actually almost like it already works: If you manually provide the chunks you can request a custom Ack message that will be delivered back to you when the spray-can layer is able to process the next part. See this example for how to stream from a spray route.

I indicate my content length is 100 bytes

A note upfront: In HTTP you don't strictly need to specify a content-length for responses because a response body can be delimited by closing the connection which is what spray does if chunkless streaming is enable. However, if you don't want to close the connection (because you would lose this persistent connection) you can now specify an explicit Content-Length header in your ChunkedResponseStart message (see #802) which will prevent the closing of the connection.

Upvotes: 2

Related Questions