CODEWITHSUNDEEP
python-3.xprotocol-buffers
Omri Shneor
Omri Shneor

Reputation: 1005

Changing protobuf optional field to oneof

I have the following message:

message Message {
    int64 id = 1;
    google.protobuf.FloatValue weight = 2;
    google.protobuf.FloatValue override_weight = 3;
}

and I wish to change the type of weight and override_weight(optional fields) to google.protobuf.DoubleValue so what I did was the fllowing:

message Message {
    int64 id = 1;
    oneof weight_oneof {
        google.protobuf.FloatValue weight = 2 [deprecated=true];
        google.protobuf.DoubleValue double_weight = 4;
    }
    oneof override_weight_oneof {
        google.protobuf.FloatValue override_weight = 3 [deprecated=true];
        google.protobuf.DoubleValue double_override_weight = 5;
    }
}

My question is, lets assume I have old messages who were compiled by the previous protobuff message compiler for the old message, would I be able to parse them as the new message? The documentation is very vague about this:

"Move optional fields into or out of a oneof: You may lose some of your information (some fields will be cleared) after the message is serialized and parsed. However, you can safely move a single field into a new oneof and may be able to move multiple fields if it is known that only one is ever set."

Has anyone tried this before? what is the best practice for this situation?

Upvotes: 7

Views: 5558

Answers (1)

Bart
Bart

Reputation: 1580

As far as I know fields in an oneof are just serialize using their tag number. The serialized data does not indicate if a field is part of an oneof. This is all handled by the serializer and deserializer. So as long as the tag numbers do not conflict it can be assumed that it will work in both directions, old messages to a new serializer and new messages to an old serializer.

You could test this using an online protobuf deserializer.

Verification: The code does indeed produce the same byte strings. Below you will find the message definitions and python code I used. The python code will output a byte string you can copy and use in the decoder of Marc Gravell.

syntax = "proto3";

message MessageA {
    int64 id = 1;
    float weight = 2;
    float override_weight = 3;
}

message MessageB {
    int64 id = 1;
    oneof weight_oneof {
        float weight = 2 [deprecated=true];
        double double_weight = 4;
    }
    oneof override_weight_oneof {
        float override_weight = 3 [deprecated=true];
        double double_override_weight = 5;
    }
}

import Example_pb2

# Set some data in the original message
msgA = Example_pb2.MessageA()
msgA.id = 1234
msgA.weight = 3.21
msgA.override_weight = 5.43

# Output the serialized bytes in a pretty format
str = 'msgA   = '
for x in msgA.SerializeToString():
    str += "{:02x} ".format(x)
print(str)

# Next set the original fields in the new message
msgB = Example_pb2.MessageB()
msgB.id = 1234
msgB.weight = 3.21
msgB.override_weight = 5.43

# Output the serialized bytes in a pretty format
str = 'msgB 1 = '
for x in msgB.SerializeToString():
    str += "{:02x} ".format(x)
print(str)

# And finally set the new fields in msgB
msgB.double_weight = 3.21
msgB.double_override_weight = 5.43

# Output the serialized bytes in a pretty format
str = 'msgB 2 = '
for x in msgB.SerializeToString():
    str += "{:02x} ".format(x)
print(str)

The output of the python script was:

msgA   = 08 d2 09 15 a4 70 4d 40 1d 8f c2 ad 40
msgB 1 = 08 d2 09 15 a4 70 4d 40 1d 8f c2 ad 40
msgB 2 = 08 d2 09 21 ae 47 e1 7a 14 ae 09 40 29 b8 1e 85 eb 51 b8 15 40

As you can see message A and message B yield the same byte string when setting the original fields. Only when you set the new fields you get a different string.

Upvotes: 7

Related Questions