How to deal with a slow consumer in traditional Java NIO?

Question

So, I've been brushing up my understanding of traditional Java non-blocking API. I'm a bit confused with a few aspects of the API that seem to force me to handle backpressure manually.

For example, the documentation on WritableByteChannel.write(ByteBuffer) says the following:

Unless otherwise specified, a write operation will return only after writing all of the requested bytes. Some types of channels, depending upon their state, may write only some of the bytes or possibly none at all. A socket channel in non-blocking mode, for example, cannot write any more bytes than are free in the socket's output buffer.

Now, consider this example taken from Ron Hitchens book: Java NIO.

In the piece of code below, Ron is trying to demonstrate how we could implement an echo response in a non-blocking socket application (for context here's a gist with the full example).

//Use the same byte buffer for all channels. A single thread is
//servicing all the channels, so no danger of concurrent access.
private ByteBuffer buffer = ByteBuffer.allocateDirect(1024);

protected void readDataFromSocket(SelectionKey key) throws Exception {
    var channel = (SocketChannel) key.channel();
    buffer.clear(); //empty buffer

    int count;
    while((count = channel.read(buffer)) > 0) {
        buffer.flip(); //make buffer readable

        //Send data; don't assume it goes all at once
        while(buffer.hasRemaining()) {
            channel.write(buffer);
        }

        //WARNING: the above loop is evil. Because
        //it's writing back to the same nonblocking
        //channel it read the data from, this code
        //can potentially spin in a busy loop. In real life
        //you'd do something more useful than this.

        buffer.clear(); //Empty buffer
    }

    if(count < 0) {
        //Close channel on EOF, invalidates the key
        channel.close();
    }
}

My confusion is on the while loop writing into output channel stream:

//Send data; don't assume it goes all at once
while(buffer.hasRemaining()) {
   channel.write(buffer);
}

It really confuses me how NIO is helping me here. Certainly the code may not be blocking as per the description of the WriteableByteChannel.write(ByteBuffer), because if the output channel cannot accept any more bytes because its buffer is full, this write operation does not block, it just writes nothing, returns, and the buffer remains unchanged. But --at least in this example-- there is no easy way to use the current thread in something more useful while we wait for the client to process those bytes. For all that matter, if I only had one thread, the other requests would be piling up in the selector while this while loop wastes precious cpu cycles “waiting” for the client buffer to open some space. There is no obvious way to register for readiness in the output channel. Or is there?

So, assuming that instead of an echo server I was trying to implement a response that needed to send a big number of bytes back to the client (e.g. a file download), and assuming that the client has a very low bandwidth or the output buffer is really small compared to the server buffer, the sending of this file could take a long time. It seems as if we need to use our precious cpu cycles attending other clients while our slow client is chewing our file download bytes.

If we have readiness in the input channel, but not on the output channel, it seems this thread could be using precious CPU cycles for nothing. It is not blocked, but it is as if it were since the thread is useless for undetermined periods of time doing insignificant CPU-bound work.

To deal with this, Hitchens' solution is to move this code to a new thread --which just moves the problem to another place--. Then I wonder, if we had to open a thread every time we need to process a long running request, how is Java NIO better than regular IO when it comes to processing this sort of requests?

It is not yet clear to me how I could use traditional Java NIO to deal with these scenarios. It is as if the promise of doing more with less resources would be broken in a case like this. What if I were implementing an HTTP server and I cannot know how long it would take to service a response to the client?

It appears as if this example is deeply flawed and a good design of the solution should consider listening for readiness on the output channel as well, e.g.:

registerChannel(selector, channel, SelectionKey.OP_WRITE);

But how would that solution look like? I’ve been trying to come up with that solution, but I don’t know how to achieve it appropriately.

I'm not looking for other frameworks like Netty, my intention is to understand the core Java APIs. I appreciate any insights anyone could share, any ideas on what is the proper way to deal with this back pressure scenario just using traditional Java NIO.

How to deal with a slow consumer in traditional Java NIO?

Answers (1)

Related Questions