protobuf and List<object> - how to serialize / deserialize?

Question

I have a List<object> with different types of objects in it like integers, strings, and custom types. All custom types are protobuf-adjusted. What I wanna do now is to serialize / deserialize this list with protobuf.net. Up until now I suspect that I have to declare each and every type explicitly, which is unfortunately not possible with these mixed-list constructs. Because the binary formater has no problems to do these things I hope that I missed something and that you can help me out. So my question is how to deal with objects in protobuf.net.

Answer 1

There is a way of doing this, albeit not a very clean way, by using a wrapper object that utilises another serialisation mechanism that supports arbitrary objects. I am presenting an example below using JSON but, like I said, resorting to a different serialisation tool seems like defeating the purpose of using protobuf:

[DataContract]
public class ObjectWrapper
{
    [DataMember(Order = 1)]
    private readonly string _serialisedContent;

    [DataMember(Order = 2)]
    private readonly string _serialisedType;

    public object Content { get; private set; }

    [UsedImplicitly]
    private ObjectWrapper() { }

    public ObjectWrapper(object content)
    {
        _serialisedContent = JsonConvert.SerializeObject(content);
        _serialisedType = content.GetType().FullName;
        Content = content;
    }

    [ProtoAfterDeserialization]
    private void Initialise()
    {
        var type = Type.GetType(_serialisedType);

        Content = type != null
            ? JsonConvert.DeserializeObject(_serialisedContent, type)
            : JsonConvert.DeserializeObject(_serialisedContent);
    }
}

EDIT: This can also be done using C#'s built-in binary serialisation

[DataContract]
public class ObjectWrapper
{
    [DataMember(Order = 1)]
    private readonly string _serialisedContent;

    public object Content { get; private set; }

    [UsedImplicitly]
    private ObjectWrapper() { }

    public ObjectWrapper(object content)
    {
        using (var stream = new MemoryStream())
        {
            var formatter = new BinaryFormatter();
            formatter.Serialize(stream, content);
            stream.Flush();
            stream.Position = 0;
            _serialisedContent = Convert.ToBase64String(stream.ToArray());
        }
    }

    [ProtoAfterDeserialization]
    private void Initialise()
    {
        var data = Convert.FromBase64String(source);
        using (var stream = new MemoryStream(data))
        {
            var formatter = new BinaryFormatter();
            stream.Seek(0, SeekOrigin.Begin);
            return formatter.Deserialize(stream);
        }
    }
}

Answer 2

List<YourClass> list;
ProtoBuf.Serializer.Deserialize<List<YourClass>>(filestream);

Answer 3

(disclosure: I'm the author of protobuf-net)

BinaryFormatter is a metadata-based serializer; i.e. it sends .NET type information about every object serialized. protobuf-net is a contract-based serializer (the binary equivalent of XmlSerializer / DataContractSerializer, which will also reject this).

There is no current mechanism for transporting arbitrary objects, since the other end will have no way of knowing what you are sending; however, if you have a known set of different object types you want to send, there may be options. There is also work in the pipeline to allow runtime-extensible schemas (rather than just attributes, which are fixed at build) - but this is far from complete.

---

This isn't ideal, but it works... it should be easier when I've completed the work to support runtime schemas:

using System;
using System.Collections.Generic;
using ProtoBuf;
[ProtoContract]
[ProtoInclude(10, typeof(DataItem<int>))]
[ProtoInclude(11, typeof(DataItem<string>))]
[ProtoInclude(12, typeof(DataItem<DateTime>))]
[ProtoInclude(13, typeof(DataItem<Foo>))]
abstract class DataItem {
    public static DataItem<T> Create<T>(T value) {
        return new DataItem<T>(value);
    }
    public object Value {
        get { return ValueImpl; }
        set { ValueImpl = value; }
    }
    protected abstract object ValueImpl {get;set;}
    protected DataItem() { }
}
[ProtoContract]
sealed class DataItem<T> : DataItem {
    public DataItem() { }
    public DataItem(T value) { Value = value; }
    [ProtoMember(1)]
    public new T Value { get; set; }
    protected override object ValueImpl {
        get { return Value; }
        set { Value = (T)value; }
    }
}
[ProtoContract]
public class Foo {
    [ProtoMember(1)]
    public string Bar { get; set; }
    public override string ToString() {
        return "Foo with Bar=" + Bar;
    }
}
static class Program {
    static void Main() {
        var items = new List<DataItem>();
        items.Add(DataItem.Create(12345));
        items.Add(DataItem.Create(DateTime.Today));
        items.Add(DataItem.Create("abcde"));
        items.Add(DataItem.Create(new Foo { Bar = "Marc" }));
        items.Add(DataItem.Create(67890));

        // serialize and deserialize
        var clone = Serializer.DeepClone(items);
        foreach (DataItem item in clone) {
            Console.WriteLine(item.Value);
        }
    }
}