CODEWITHSUNDEEP
goprotocol-buffersgrpc
Julio Guerra
Julio Guerra

Reputation: 5661

Unbuffered bidirectional data streaming with gRPC: how to get the size of the client-side buffer?

I am streaming data from a server to a client and I would like the server not to read and send more data than the client's buffer size.

Given:

service StreamService {
  rpc Stream(stream Buffer) returns (stream Buffer);
}

message Buffer {
  bytes data = 1;
}

My client's program basically looks like: 

func ReadFromServer(stream StreamService_StreamClient, buf []byte) (n int, err error) {
  // I actually don't need more than len(buf)...
  // How could I send len(buf) while stream is bidirectional...?
  buffer, err := stream.Recv()
  if err != nil {
     return 0,  err
  }
  n = copy(buf, buffer.Data)
  // buf could also be smaller than buffer.Data...
  return n, nil
}

So how could I send len(buf) while the RPC's stream is bidirectional, i.e. the send direction is used by another independent stream of data? Note that I don't use client or server-side buffering to avoid loosing data when one of them is terminated (my data-source is an I/O).

Upvotes: 3

Views: 3815

Answers (2)

Vitaly Isaev
Vitaly Isaev

Reputation: 5815

I think there's no built-in solution for that. The use-case looks little bit weird: why server must care about client's state at all? If it really needs to, you should extend your bidirectional stream: the client must request byte slices of a particular size (according to the own buffer size and other factors).

By the way, you may find useful message size limit settings GRPC client and server: https://godoc.org/google.golang.org/grpc#MaxMsgSize https://godoc.org/google.golang.org/grpc#WithMaxMsgSize

Upvotes: 0

Eric Anderson
Eric Anderson

Reputation: 26434

gRPC provides no mechanism for this. It only provides push-back when a sender needs to slow down. But there will still be buffering happening internally and that is not exposed because gRPC is message-based, not byte-based.

There's really only two options in your case:

  1. Server chunks responses arbitrarily. The client Recv()s when necessary and any extra is manually managed for later.
  2. The client sends a request asking for a precise amount to be returned, and then waits for the response.

Note that I don't use client or server-side buffering to avoid loosing data when one of them is terminated (my data-source is an I/O).

This isn't how it works. When you do a Send() there is no guarantee it is received when the call returns. When you do a Recv() there is no guarantee that the message was received after the recv call (it could have been received before the call). There is buffering going on, period.

Upvotes: 3

Related Questions