Serializing with ProtoBuf.NET without tagging members

Question

I've read somewhere a comment by the author of ProtoBuf.NET that:

There are options to automatically infer the numbers, but that is brittle and not recommended. Only use this if you know you never need to add more members (it orders them alphabetically, so adding a new AardvarkCount will break everything).

This is exactly that sort of situation I am interested in :)

I have something that is akin to a map-reduce scenario where I want to serialize results generated on remote machines using protocol buffers (e.g. the "map" side of map-reduce) and later read them and combine those results for further processing (e.g. the "reduce" side).

I don't want to start an attribute decoration marathon over every possible class I have that might get serialized during this process, and I do find the protocol buffers to be very alluring as I can create result with Mono and consume them effortlessly on MS.NET and vice-versa...

The apparent downsides of not pre-tagging the members doesn't bother me as exactly the same software revision does generation/consumptionn, so I don't need do worry about new members popping up in the code and messing my whole scheme...

So in short, my question is:

• How do I do it (Serialize with ProtoBuf.NET without tagging/building Meta classes on my own)?
• Is there any hole in my scheme that I've glaringly missed?

Answer 1

I had the same problem and that how I've resolved it with TypeModel. It's based on properties ordered by their name (however it doesn't check on property setter/getter existence or serialize-ability of a given type):

[TestFixture]
public class InferredProtoPoc
{
    [Test]
    public void UsageTest()
    {
        var model = TypeModel.Create();
        // Dynamically create the model for MyPoco
        AddProperties(model, typeof(MyPoco));
        // Display the Generated Schema of MyPoco
        Console.WriteLine(model.GetSchema(typeof(MyPoco)));

        var instance = new MyPoco
        {
            IntegerProperty = 42,
            StringProperty = "Foobar",
            Containers = new List<EmbeddedPoco>
            {
                new EmbeddedPoco { Id = 12, Name = "MyFirstOne" },
                new EmbeddedPoco { Id = 13, Name = "EmbeddedAgain" }
            }
        };

        var ms = new MemoryStream();
        model.Serialize(ms, instance);
        ms.Seek(0, SeekOrigin.Begin);
        var res = (MyPoco) model.Deserialize(ms, null, typeof(MyPoco));
        Assert.IsNotNull(res);
        Assert.AreEqual(42, res.IntegerProperty);
        Assert.AreEqual("Foobar", res.StringProperty);
        var list = res.Containers;
        Assert.IsNotNull(list);
        Assert.AreEqual(2, list.Count);
        Assert.IsTrue(list.Any(x => x.Id == 12));
        Assert.IsTrue(list.Where(x => x.Id == 12).Any(x => x.Name == "MyFirstOne"));
        Assert.IsTrue(list.Any(x => x.Id == 13));
        Assert.IsTrue(list.Where(x => x.Id == 13).Any(x => x.Name == "EmbeddedAgain"));
    }

    private static void AddProperties(RuntimeTypeModel model, Type type)
    {
        var metaType = model.Add(type, true);
        foreach (var property in type.GetProperties().OrderBy(x => x.Name))
        {
            metaType.Add(property.Name);
            var propertyType = property.PropertyType;
            if (!IsBuiltinType(propertyType) &&
                !model.IsDefined(propertyType) && 
                propertyType.GetProperties().Length > 0)
            {

                AddProperties(model, propertyType);
            }
        }
    }

    private static bool IsBuiltinType(Type type)
    {
        return type.IsValueType || type == typeof (string);
    }
}

public class MyPoco
{
    public int IntegerProperty { get; set; }
    public string StringProperty { get; set; }
    public List<EmbeddedPoco> Containers { get; set; }
}

public class EmbeddedPoco
{
    public int Id { get; set; }
    public String Name { get; set; }
}

And that's what you get from running it.

    message EmbeddedPoco {
       optional int32 Id = 1;
       optional string Name = 2;
    }
    message MyPoco {
       repeated EmbeddedPoco Containers = 1;
       optional int32 IntegerProperty = 2;
       optional string StringProperty = 3;
    }

For performance, you could opt to compile the TypeModel, and/or store the generated proto for future uses. Beware however that hidden dependency on Protocol Buffer could be dangerous in the long run if the poco (Plain old container object) evolves.

Answer 2

If you can live with a single attribute, then the trick is:

    [ProtoContract(ImplicitFields = ImplicitFields.AllPublic)]
    public class WithImplicitFields
    {
        public int X { get; set; }
        public string Y { get; set; }
    }

there are 2 options here; AllPublic works like XmlSerializer - public properties and fields are serialized (using the alphabetic order to choose tag numbers); AllFields works a bit like BinaryFormatter - the fields are serialized (again, alphabetic).

I can't remember if this is yet available on the v2 API; I know it is on my list of things to ensure work! But if you want it in v2 without any attributes, I'm sure I can add an Add(ImplicitFields) overload.

As long as the 2 ends are never out of step, this is fine. If you store the data, or don't version the two ends "in step", then there could be problems. See also the intellisense comments on the enum (which pretty much repeats the warning that you are already aware of).