CODEWITHSUNDEEP
c#protobuf-net
allmhuran
allmhuran

Reputation: 4464

Protobuf-net pattern for optionally assignable nullable types (not collections)

I have been searching for a while, and have found the various threads indicating that protobuf-net can handle nullable types, the various threads about things like empty collections (not relevant here), and the default value behaviour for what would previously have been "optional" fields in proto2, but I have not found a specific answer to the following question. This will be my first time using protobuf-net, or protocol buffers as a serialization format.

Imagine I am sending messages between multiple different applications using an event carried state transfer pattern - so publishers and subscribers of messages containing state - and some kind of shared model for that state. Any participating applications have some representation of "the same" entity (hence the ECST), but not all systems understand all attributes. In terms of persistence in the applications' SQL database, that might look like this (scales left off for brevity):

table App1Products { productKey int, productName varchar null }

table App2Products { productKey int, productName varchar null, productWeightKg decimal null }

table App3Products { productKey int }

For the sake of the example, suppose the trivial shared model is the union of all of the distinct attributes: { productKey, productName, productWeightKg }

Now suppose someone updates a productName in App1Products. We want to publish that changed state. When we do this we can't populate the entire shared model, because App1 does not include the productWeightKg in its schema. We need to somehow "leave out" any value for this element in a way that potential consumers understand that it was not populated.

We can't merely send a default value (0) (or let subscribers deserialize a missing element to a default value) to convey "no update" semantics, because that would cause the productWeightKg value to be set to 0 in the App2Products table when the message is received by App2. We can't send null to convey "no update" semantics, because null is a legitimate value for the column as well.

Ultimately, we need the subscriber code at App2 to construct an update statement such that either the productWeightKg column is not referenced, or it is simply set to itself, and we need some way of telling App2's subscriber code to do that.

One solution seems to be to create an additional element in the message for each field, indicating whether the field is or is not set. In terms of the message content, we might use something like this:

[ProtoContract]
public class Product
{
    [ProtoMember(1)]
    public int ProductKey { get; set; }

    [ProtoMember(2)]
    public string productName { get; set; }

    [ProtoMember(3)]
    public decimal? productWeightKg { get; private set; }

    [ProtoMember(4)]
    public bool productWeightKgSet { get; private set; }

    public void SetProductWeight(decimal? weight)
    {
        productWeightKg = weight;
        productWeightKgSet = true;
    }

    public void ClearProductWeight()
    {
        productWeightKgSet = false;
    }
}

If that is a reasonable sort of pattern to use, then the next "obvious" idea would be to create some kind of template class for this sort of behaviour which we can reuse for all of our message classes...

public class Optional<T>
{
    public T Value { get; private set; }
    public bool HasValue { get; private set; }

    public void Set(T val) { Value = val; HasValue = true; }
    public void Clear() { HasValue = false; Value = default; }
}

Is this a reasonable sort of approach to solving this problem, or is there some other "known good pattern" that I've missed, or something about this pattern which won't work as expected with protobuf-net?

My limited understanding at the moment is that this might require that the Optional class be decorated with all of its possible child implementations using the ProtoInclude attribute, is that correct?

Upvotes: 3

Views: 684

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1062975

Ultimately, the intention of protobuf-net is not to provide a robust field-tracking mechanism, and since it works against POCO types - it has nowhere to store any additional state, except that which your object model provides. It does support considitonal serialization, and there are many things that your model can do to track changes internally, as discussed here; this may be useful in combintion with Merge (rather than deserialization) - but going beyond that isn't something provided out-of-the-box (it also isn't provided by most other POCO serializers, as far as I know).

There is some crossover between what you describe and the FieldMask concept, but: to date, protobuf-net has not has a need to implement or support FieldMask.

I'm always happy to explore new things that the library might be able to do to help people, but if there is a missing library feature: this is probably something better discussed on GitHub, along with very specific details of the target scenario and motivations, etc.

Upvotes: 1

Related Questions