CODEWITHSUNDEEP
protobuf-net
Manish
Manish

Reputation: 1786

Update protobuf after it has been serialized

We are saving protobufs inside database tables as Byte[]s. However, we are running into a situation where we have to update the protobufs after they have been saved. This is creating two problems - (1) we have to serialize/ deserialize them to update them and save them back and (2) we have to replicate the protobuf hierarchy in the update code (if obj.getType() == typeOf(A)) then ... else (if type == ) do ...

Is there a way to modify the Byte[] without having to serialize/ deserialize and identifying the protobuf type?

Thanks Manish

Upvotes: 1

Views: 623

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1064104

The protocol buffers format is generally used as an opaque format, meaning: you wouldn't expect to know the insides - the typical scenario is that you would just load it, make your changes, and persist it. Fortunately the format is terse and efficient, such that this is rarely a problem at all.

As a comparison, you wouldn't have any expectation of editing BinaryFormatter data without using BinaryFormatter, and most people would still use the serializer to make changes to xml / json, because the format is sufficiently complex that it isn't worth trying to write vanilla code that handles it robustly.

However! If you are determined to work on protobuf data without the serializer, you can also use the reader/writer API. This, like with XmlReader/XmlWriter, assumes you know quite a bit about the underlying specification, and you would still need to know the field numbers that you are interested in (although you wouldn't need to know much about the fields that you aren't interested in).

If you had a very specific (i.e. runnable) example I could probably show you how to do this. However! I will advise that the simplest thing is probably to have the model around.

I am, however, a bit unclear why you need to "replicate the protobuf hierarchy". Protobuf-net, in the default usage, does not care one bit what the actual types are (as long as they meet the contract). If the scenario is that you are making changes to the DTOs, then:

  1. yes, doing dangerous things like making breaking changes to a DTO can impact things - if it hurst, stop doing it (there are usually equally convenient but safe ways to change a DTO over time)
  2. in many cases you can use the RuntimeTypeModel features of v2 to shim over the differences between 2 versions of a model - by having two different RuntimeTypeModel configurations (perhaps one of them involving surrogates, or using different properties, etc)

I would need to see the problem you are trying to solve to advise further, though.

Upvotes: 1

Related Questions