CODEWITHSUNDEEP
c++gzipprotocol-buffers
abyss.7
abyss.7

Reputation: 14462

How to read the message of a specific size from `GzipInputStream` in protobuf in C++?

I want to read the message of a specific size from a network socket. The size is encoded before the message it-self. The size + message are gzipped with GzipOutputStream.

Here goes the simplified sample code of my decoder:

unsigned size;
GzipInputStream gzip_stream(&file_input_stream_, GzipInputStream::ZLIB);
{
  CodedInputStream coded_stream(&gzip_stream);
  coded_stream.ReadVarint32(&size);
}

message->ParseFromBoundedZeroCopyStream(&gzip_stream, size);

In the documentation on MessageLite::ParseFromBoundedZeroCopyStream() there are contrary statements:

So, I'm confused - will this function read size bytes from the stream, or will it expect the message of size bytes? - I suggest the former, since I can't properly read two messages consequently from the same socket.

Question:

How to properly read the message of a specific size (SerializeAsString().size()) from GzipInputStream?

Upvotes: 1

Views: 676

Answers (1)

Kenton Varda
Kenton Varda

Reputation: 45296

The bullet points are not contradictory. Since ParseFromBoundedZeroCopyStream doesn't know what kind of stream it's reading from, the documentation is of course referring to the size of the data that it gets out of the stream, post-decompression. It will read exactly size uncompressed bytes from the stream, interpreting them as a single message. If you only know the compressed size of your message then you cannot use this method; you'll have to do something more complicated, e.g. layer the GzipInputStream on top of a LimitingInputStream.

(Note also that if the method returns false to indicate failure, then it may have stopped reading early, leaving your stream in an indeterminate state. If this is a problem, you'll have to do something more complicated to make sure you read the full size even in case of parse error, such as setting up a LimitingInputStream yourself and explicitly draining it of data after a parse failure.)

Upvotes: 1

Related Questions