CODEWITHSUNDEEP
javagrpcgrpc-java
Yury Vasyutinskiy
Yury Vasyutinskiy

Reputation: 63

Java gRPC server for long-lived streams effective implementation

I would like to understand a part of gRPC framework for resource management for long-lived streams. Let's say we have infinite source of rare (once a second or so) events that we want to stream to clients by the means of grpc stream. The events are generated by a single application thread on the server.

I see two possible implementations to stream events:

  1. Spin in the caller thread within rpc call and communicate with source via (blocking) queue
  2. Expose StreamObserver to event generating thread(s) and populate all client streams from there.

Option one seems straightforward but a bit heavy on thread count - one thread per client for sparse stream seems like an overkill. Each thread takes some heap occupies scheduler and so on.

Option two looks a bit more resource-friendly. However I couldn't find any materials on internet to support this approach. I am not sure that gRPC server is not going to close ServerCall or Context unexpectedly leading to stream abrupt close. Or there could be some other side effects I am not aware of.

So my questions are: What is the recommended way of implementing long-lived streams? Are there any other possible approaches to implement the described problem. Is option 2 legit or one should stick with 1 client 1 thread approach?

I've tried to create a prototype with option two, and it seems to be working. But I still would like to get the answers.

Upvotes: 2

Views: 2411

Answers (1)

Eric Anderson
Eric Anderson

Reputation: 26394

Both approaches are fine from gRPC's perspective. You are free to stick with 1 client, 1 thread approach when it is convenient. For streaming cases it is generally best to avoid spinning in the caller thread, but you can use a second thread to send instead; that's quite normal. On the other hand, passing the StreamObservers to a single thread for management has resource benefits, and is a good approach too.

You should consider how to respond to slow clients, when events are generated faster than they are being sent (i.e., flow control).

You will want to cast the provided StreamObserver to ServerCallStreamObserver to access additional APIs. It provides setOnReadyHandler(Runnable) and isReady() for detecting slow clients. gRPC Java allows you to call onNext(...) even when not ready, but doing so will buffer.

The on-ready handler is a callback that uses the same thread as the caller thread, so if you spin in the caller thread you won't be able to receive that callback. This is why for streaming it is generally best to avoid spinning in the caller thread.

Using a dedicated sending thread has the advantage of having a clear queue and separation between producer and consumer. You can choose a queue size and decide what to do when that queue is full. Directly interacting with the StreamObservers in one thread has less resource usage. The complexity of both options varies. The "correct" choice of approach is based on scale, resource considerations, specifics of your service, and your preferences.

Upvotes: 2

Related Questions