CODEWITHSUNDEEP
protocol-buffersflatbuffers
Ibraim Ganiev
Ibraim Ganiev

Reputation: 9390

Is there alternative to flatbuffers struct in protobuf?

The question is about protobuf, but first I want to explain similar concept from flatbuffers:

I know that in flatbuffers I can create Point2d structure with scalar-types like this:

struct Point2d {
  x: double;
  y: double;
}

table PointCloud {
  point_array: [Point2d] (required);
}

Struct here sacrifices schema evolution, and I can live with this, but in return I can have array of such points with millions of elements, with almost-0 overhead, each point struct (which is 16 bytes-long) will be written into buffer one after another, so 2 points will take 32 bytes, 4 points 64 bytes, etc.

I am new to protobuf world, but it seems like protobuf ALWAYS writes varint with field-id into the output stream, even when "message" is repeated and contains scalar types. So for each repeated point2d (16 bytes) we have 2 additional varint bytes, which are completely useless in "array".

Am I right about this? Is it going to actually write varint bytes before each x and y field of each repeated point in point_array? Is there alternative to flatbuffer struct in protobuf?

To be more specific I include proto that is similar to flatbuffer file:

message Point2d {
  double x = 1;
  double y = 2;
}

message PointCloud {
  // Is protobuf serializing 2 varints with each Point2d in this repeated field?
  repeated Point2d point_array = 1;
}

Upvotes: 0

Views: 462

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1062895

In protobuf, you cannot avoid the field headers in structured data, but: if you use an unstructured simple array, it can be "packed" (which just means: without field headers). This is automatic in proto3 when using a "repeated" primitive such as int32. In proto2, you need to opt in using the packed marker.

Upvotes: 2

Related Questions