CODEWITHSUNDEEP
protocol-buffers
user583933
user583933

Reputation: 119

Best practice for determining the type of a protocol buffers message

I need to serialize and deserialize a sequence of protocol buffers messages to and from a byte stream. There are a handful of predetermined message types. What is the recommended way of encoding the type information so that my application can know which type it should be reading?

Upvotes: 7

Views: 3827

Answers (2)

jpa
jpa

Reputation: 12156

The most common way to do this is to use a union message.

For example:

message AnyMessage {
    optional Message1 msg1 = 1;
    optional Message2 msg2 = 2;
    ...
}

Then all messages are encoded/decoded inside an AnyMessage container. Starting with protobuf 2.6, you can also use the oneof specifier which will ensure that only one of the submessages is set.

Upvotes: 9

Bruce Martin
Bruce Martin

Reputation: 10543

My suggestions in no particular order are:

  1. Include a Field that holds the proto name / id (and give it the same field number in all proto's say 1)

  2. Use self describing message (at the bottom of the page). In this case you can either

    • include the FileDescriptorSet as a field (in your messages)
    • In java (and some other implementations) you can write the FileDescriptor set as the first message in a delimited file.

  3. Keep the proto names small and use proto-name in the File name e.g.

    salesProto_Store001.bin

    This has several good points:

    • It is immediate obvious what the file format is
    • You can scan shell scripts to find where the proto is used.
    • This technique can be used by itself or combined with the above 2.
    • This technique can be used for any schema based files (e.g. Cobol).

Finally the ProtobufEditor

  • Supports Self Describing messages where

    • The File descriptor is the first message in a delimited file
    • The first field in the message

  • Has a search function which will try an match the fields in a Protobuf message against known Proto definition files and gives you possible match's

Background: In case you did not know, protocol buffers proto file can be converted to a FileDescriptorSet protocol buffer message and stored

Self describing messagew:

message SelfDescribingMessage {
  // Set of .proto files which define the type.
  required FileDescriptorSet proto_files = 1;

  // Name of the message type.  Must be defined by one of the files in
  // proto_files.
  required string type_name = 2;

  // The message data.
  required bytes message_data = 3;
}

Upvotes: 2

Related Questions