CODEWITHSUNDEEP
javaprotocol-buffersdelimited
Brunaldo
Brunaldo

Reputation: 3

How to fully read a file with delimited messages in Google Protobufs?

I'm trying to read a file, which has multiple delimited messages in it (in the thousands), how can I do this properly using Google protobufs?

This is how I'm writing the delimited:

MyMessage myMessage = MyMessage.parseFrom(byte[] msg);
myMessage.writeDelimitedTo(FileOutputStream);

and this is how I'm reading the delimited file;

    CodedInputStream is = CodedInputStream.newInstance(new FileInputStream("/location/to/file"));

    while (!is.isAtEnd()) {
        int size = is.readRawVarint32(); 
        MyMessage msg = MyMessage.parseFrom(is.readRawBytes(size));
    //do stuff with your messages
    }

I'm kind of confused because the accepted answer in this question say's to use .parseDelimitedFrom() to read the delimited bytes; Google Protocol Buffers - Storing messages into file

However, when using .parseDelimitedFrom(), it only reads the first message. (I don't know how to read the whole file using parseDelimitedFrom()).

This comment say's to write the delimited messages using CodedOutputStream: Google Protocol Buffers - Storing messages into file (i.e. writer.writeRawVariant()). I'm currently using the implementation of this comment to read the whole file. Does writeDelimitedTo() basically do the same thing as 

writer.writeRawVarint32(bytes.length);

and

 writer.writeRawBytes(bytes);

Also, if my way isn't the proper way of reading a whole file consisting of delimited messages, can you please show me what is?

thank you.

Upvotes: 0

Views: 3850

Answers (1)

Kenton Varda
Kenton Varda

Reputation: 45151

Yes, writeDelimitedTo() simply writes the length as a varint followed byte the bytes. There's no need to use CodedOutputStream directly if you're working in Java.

parseDelimitedFrom() parses one message, but you may call it repeatedly to parse all the messages in the InputStream. The method will return null when you reach the end of the stream.

Upvotes: 3

Related Questions