CODEWITHSUNDEEP
netty
Michael
Michael

Reputation: 4886

how to stream a response over HTTP with netty

I'm using Netty 3.6.6 and I'd like to send a large response back to the caller. I can't copy the response body into a ChannelBuffer as in some cases it will be very large.

I'm migrating a server from CXF to Netty, previously, I could just use the OutputStream provided by CXF to write the data.

I originally tried to just send the response without content, and then continued to write data to the Channel in a series of 8k buffers. This failed as the client seemed to get the original response and see no data and complain. I tried setting the response as chunked, but this didnt seem to make a difference, nor did setting the chunked header, the client always saw an empty stream.

I saw the file server example for 3.6.6, and that's similar to what I want to do, except the data will not be a file. I saw the ChunkedStream & NioStream, which seemed close to what I need, except they take InputStream/ReadableByteChannel whereas I have an OutputStream; I could try using the PipedInput & OutputStreams, but that seems like it would introduce an unfortunate bottleneck.

I'm sure there's a way to stream a lot of data back to the client in response to a request, but I'm just not seeing how to do it unless I have a file.

I am also curious how you let the client know the response is complete if the connection is keep-alive, and you're streaming content, but don't know the content length. Seems like the client would wait forever for the connection to close in those cases.

Modifying the static file server example from 3.6.6 to remove the content-length header (just comment it out), specify that its a chunked response

   response.setChunked(true);
   response.setHeader(Names.TRANSFER_ENCODING, Values.CHUNKED);

and then using ChunkedNioStream to send the file after writing the response:

    // Write the initial line and the header.
    ch.write(response);

    final ReadableByteChannel aIn = java.nio.channels.Channels.newChannel(new FileInputStream(file));
    ChannelFuture writeFuture = ch.write(new ChunkedNioStream(aIn));

Yields the undesired behavior, the client gets a couple hundred bytes and then stop receiving, basically what I'm seeing in my application. The correct thing only seems to happen with a content-length, which is not feasible in my use case.

Upvotes: 3

Views: 5676

Answers (1)

trustin
trustin

Reputation: 12351

When you attempt to write a ChunkedNioStream to ChunkedWriteHandler, it merely produces a stream which contains the content of the ChunkedNioStream only. That is, it produces ChannelBuffers rather than HttpChunks.

Because HttpMessageEncoder handles only HttpMessage and HttpChunk, ChannelBuffer produced by ChunkedNioStream is bypassed to the wire, without HTTP chunk header prepended, causing your browser confused.

To fix this problem, you have to implement your own ChunkedInput which produces HttpChunks instead of ChannelBuffers. However, I must agree that this might be a challenging task, so you might just want to fork HttpMessageEncoder so that it also understands ChannelBuffer and treats it just like HttpChunk. Please take a look at this part of HttpMessageEncoder for more information.

Upvotes: 5

Related Questions