CODEWITHSUNDEEP
c++protocol-buffers
Chris
Chris

Reputation: 615

Overwriting a repeated field in protobuf MergeFrom vs. MergeFromString

Suppose I have a proto defined as:

MyProto {
    optional MyWrapper wrapper = 1;
}

where:

MyWrapper {
    repeated int32 data = 1;
}

When I call MergeFromString on two text specifications of MyProto, the two versions of the repeated field inside the wrapper are concatenated (one is appended to the other.) I really want them to be overwritten instead. The docs for MergeFromString say:

When we find a field in |serialized| that is already present in this message:

  • If it's a "repeated" field, we append to the end of our list.
  • Else, if it's a scalar, we overwrite our field.
  • Else, (it's a nonrepeated composite), we recursively merge into the existing composite.

Clearly, with the wrapper, we're talking about the third case. So we recursively merge, and on the next go-around we see a repeated field and the values get appended to the target. So I see why this happened.

Compare this to the specification for MergeFrom:

This method merges the contents of the specified message into the current message. Singular fields that are set in the specified message overwrite the corresponding fields in the current message. Repeated fields are appended. Singular sub-messages and groups are recursively merged.

In that case, isn't the wrapper field a singular field, and wouldn't the wrapper be overridden?

So my question is two-fold,

1) Is this inconsistent, or have I misunderstood something?

2) How can I get my desired behavior of overwriting instead of merging a repeated field when I call MergeFromString?

Upvotes: 3

Views: 9747

Answers (2)

faizan
faizan

Reputation: 690

I implemented StrictMerge to get around this problem. The solution is simple, you just clear any unwanted repeated field from the target proto. But, we clear only if see conflict with corresponding field in source message.

void StrictMerge(const Message& source, Message* target) {
  ClearRepeatedField(source, target);
  target->MergeFrom(source);
}

ClearRepeatedField recursively iterates through source and target. At each level, it would just clear a repeated field in target, if there is a conflicting repeated field in source.

void ClearRepeatedField(const Message& source, Message* target) {
  const Descriptor* source_descriptor = source.GetDescriptor();
  const Reflection* source_reflection = source.GetReflection();
  const Descriptor* target_descriptor = target->GetDescriptor();
  const Reflection* target_reflection = target->GetReflection();
  for (int i = 0; i < source_descriptor->field_count(); i++) {
    const FieldDescriptor* source_field = source_descriptor->field(I);
    const FieldDescriptor* target_field = target_descriptor->field(I);
    if (source_field->is_map()) {
      // Do nothing for a map
      continue;
    }
    if (source_field->is_repeated()) {
      // Clear only if source field is not empty
      if (source_reflection->FieldSize(source, source_field) > 0) {
        target_reflection->ClearField(target, target_field);
      }
      continue;
    }
    bool has_message_value =
      source_field->type() == FieldDescriptor::TYPE_MESSAGE &&
      source_reflection->HasField(source, source_field);
      if (has_message_value) {
        const Message& source_message = source_reflection->GetMessage(
          source, source_field);
        Message* target_message = target_reflection->MutableMessage(
        target, target_field);
        // Drop repeated fields from this field's target
        ClearRepeatedField(source_message, target_message);
      }
    }
}

Upvotes: 3

Adam Cozzette
Adam Cozzette

Reputation: 1476

For the first part of your question, the MergeFrom specification you quoted is technically correct although worded a bit confusingly. It says that "singular fields" are overwritten but "singular sub-messages" are recursively merged, and your wrapper would be considered a singular sub- message.

To get the behavior you want, you should be able to use FieldMaskUtil. In particular you can call FieldMaskUtil::MergeMessageTo(...) and pass a MergeOptions configured to replace repeated fields instead of concatenating them. To do this you will first have to parse the two messages from their text format representation.

Upvotes: 5

Related Questions