CODEWITHSUNDEEP
c++serializationflatbuffers
fluter
fluter

Reputation: 13796

Is the size prefix for each buffer or entire program?

I'm streaming flat buffers over network, so each buffer is size prefixed, by FinishSizePrefixedFoo(fbb, msg), but I notice this size prefix is not the size of the message structure msg, it is the size of the whole buffer builder. For example:

flatbuffers::FlatBufferBuilder fbb;
flatbuffers::Offset<Message> msg;
msg = ns::CreateMessage(fbb, ...);
FinishSizePrefixedMessageBuffer(fbb, msg);
flatbuffers::uoffset_t len = flatbuffers::ReadScalar<flatbuffers::uoffset_t>(fbb.GetBufferPointer());
std::cout << "Buffer size is " << fbb.GetSize() << " size prefix is " << len << std::endl;
flatbuffers::Offset<Message> msg2;
msg2 = ns::CreateMessage(fbb, ...);
FinishSizePrefixedMessageBuffer(fbb, msg2);
len = flatbuffers::ReadScalar<flatbuffers::uoffset_t>(fbb.GetBufferPointer());
std::cout << "Buffer size is " << fbb.GetSize() << " size prefix is " << len << std::endl;

The output is:

Buffer size is 84 size prefix is 80
Buffer size is 168 size prefix is 164

But the expected output is:

Buffer size is 84 size prefix is 80
Buffer size is 168 size prefix is 80

Otherwise there is no way to separate the two messages.

I'm using this size prefix for verifier and deserialisation, e.g.:

buf[4] = recv(4);
uoffset_t len = ReadScalar(buf);
buf[4..len+4] = recv(len);
msg = GetSizePrefixedMessage(buf);

When I read the size is 164, a message of size 164 is expected to follow, while in fact is is two messages of 80 bytes each. Is this flatbuffers' bug?

Upvotes: 0

Views: 1544

Answers (1)

Aardappel
Aardappel

Reputation: 6074

Please make sure you develop with asserts on, as that would have already told you that the above code is not correct.

When you call Finish on a builder, you cannot serialize more data into it, unless you call Clear. So in this case the second buffer also contains a copy of the first buffer. The assert would have told you that.

Of course, you can also use 2 FlatBufferBuilder instances, if you want to easily keep both buffers available.

Upvotes: 2

Related Questions