How to handle deserialization backward compatibility for changed classes based on generic collections?

If the old version of a c++/CLI application serialized a class Foo derived from a Dictionary involving keys of type X, and the new version needs to change the key type to Z instead, then how can I best enable the application to support reading of old serialized data (still based on X) as well as new serialized data (based on Z)?

If the old situation is like this:

ref class Foo: Generic::Dictionary<X^, Y^>, ISerializable
{
 public:
  Foo(SerializationInfo^ info, StreamingContext context)
  {
     info->AddValue("VERSION", 1);
     __super::GetObjectData(info, context);
  }

  virtual void GetObjectData(SerializationInfo^ info, StreamingContext context)
     : Generic::Dictionary<X^, Y^>(info, context)
  {
     int version = info->GetInt32("VERSION");
     /* omitted code to check version, act appropriately */
  } 
}

then in the new situation I'd like to do something like this:

ref class Foo: Generic::Dictionary<Z^, Y^>, ISerializable
{
 public:
  Foo(SerializationInfo^ info, StreamingContext context)
  {
    info->AddValue("VERSION", 2);
    __super::GetObjectData(info, context);
  }

  virtual void GetObjectData(SerializationInfo^ info, StreamingContext context)
  {
     int version = info->GetInt32("VERSION");
     if (version == 1)
     {
        Generic::Dictionary<X^, Y^> old 
          = gcnew Generic::Dictionary<X^, Y^>(info, context);
        /* code here to convert "old" to new format,
           assign to members of "this" */
     }
     else
     {
        Generic::Dictionary<Z^, Y^)(info, context);
     }
  }
}

but that fails with compilation errors of type :

error C2248: 'System::Collections::Generic::Dictionary<TKey,TValue>::Dictionary' : cannot access protected member declared in class 'System::Collections::Generic::Dictionary<TKey,TValue>' with [ TKey=X ^, TValue=Y ^ ].

In simpler cases, I can use info->GetValue to extract and process individual data members, but in the current case the serialization of the Dictionary was left to .NET (through the __super::GetObjectData call) and I don't know how to use info->GetValue to extract the old Dictionary.

A related question: If I want to rename Foo to BetterFoo and yet be able to support reading old serialized data (still based on Foo) as well as new serialized data (based on BetterFoo), then how can I best do that?

I've looked into SerializationBinder and ISerializationSurrogate but couldn't figure out how to use them to solve my problems.

Upvotes: 3

Answers (1)

Louis Strous

Reputation: 1074

I've found a partial answer to my own questions. Inspection of the MemberNames and MemberValues properties of the SerializationInfo in the debugger shows the types of the members stored in there. A Dictionary<X^, Y^> is included in the SerializationInfo as an item with name KeyValuePairs and type array<System::Collections::Generic::KeyValuePair<X^, Y^>> ^. With this information, the SerializationInfo's GetValue method can be used to retrieve key-value pairs, and then they can be transformed and added to the object being filled. A SerializationBinder can be used to have the deserialization constructor of one class handle also the deserialization of another class, thus allowing backward compatibility after renaming a class. The following code shows all of these things.

using namespace System;
using namespace System::IO;
using namespace System::Collections::Generic;
using namespace System::Runtime::Serialization;

typedef KeyValuePair<int, int> Foo1kvp;
[Serializable]
public ref class Foo1: Dictionary<int, int>, ISerializable
{
public:
    Foo1() { }
    virtual void GetObjectData(SerializationInfo^ info, StreamingContext context) override
    {
        info->AddValue("VERSION", 1);
        __super::GetObjectData(info, context);
    }
    Foo1(SerializationInfo^ info, StreamingContext context)
    {
        array<Foo1kvp>^ members = (array<Foo1kvp>^) info->GetValue("KeyValuePairs", array<Foo1kvp>::typeid);
        for each (Foo1kvp kvp in members)
        {
            this->Add(kvp.Key, kvp.Value);
        }
        Console::WriteLine("Deserializing Foo1");
    }
};

typedef KeyValuePair<String^, int> Foo2kvp;
[Serializable]
public ref class Foo2: Dictionary<String^, int>, ISerializable
{
public:
    Foo2() { }
    virtual void GetObjectData(SerializationInfo^ info, StreamingContext context) override
    {
        info->AddValue("VERSION", 2);
        __super::GetObjectData(info, context);
    }
    Foo2(SerializationInfo^ info, StreamingContext context)
    {
        int version = info->GetInt32("VERSION");
        if (version == 1)
        {
            array<Foo1kvp>^ members = (array<Foo1kvp>^) info->GetValue("KeyValuePairs", array<Foo1kvp>::typeid);
            for each (Foo1kvp kvp in members)
            {
                this->Add(kvp.Key.ToString(), kvp.Value);
            }
            Console::WriteLine("Deserializing Foo2 from Foo1");
        }
        else
        {
            array<Foo2kvp>^ members = (array<Foo2kvp>^) info->GetValue("KeyValuePairs", array<Foo2kvp>::typeid);
            for each (Foo2kvp kvp in members)
            {
                this->Add(kvp.Key, kvp.Value);
            }
            Console::WriteLine("Deserializing Foo2");
        }
    }
};

ref class MyBinder sealed: public SerializationBinder
{
public:
    virtual Type^ BindToType(String^ assemblyName, String^ typeName) override
    {
        if (typeName == "Foo1")
            typeName = "Foo2";
        return Type::GetType(String::Format("{0}, {1}", typeName, assemblyName));
    }
};

int main(array<System::String ^> ^args)
{
    Console::WriteLine(L"Hello World");
    Foo1^ foo1 = gcnew Foo1;
    foo1->Add(2, 7);
    foo1->Add(3, 5);

    IFormatter^ formatter1 = gcnew Formatters::Binary::BinaryFormatter(); // no translation to Foo2
    IFormatter^ formatter2 = gcnew Formatters::Binary::BinaryFormatter();
    formatter2->Binder = gcnew MyBinder; // translate Foo1 to Foo2
    FileStream^ stream;
    try
    {
        // serialize Foo1
        stream = gcnew FileStream("fooserialized.dat", FileMode::Create, FileAccess::Write);
        formatter1->Serialize(stream, foo1);
        stream->Close();

        // deserialize Foo1 to Foo1
        stream = gcnew FileStream("fooserialized.dat", FileMode::Open, FileAccess::Read);
        Foo1^ foo1b = dynamic_cast<Foo1^>(formatter1->Deserialize(stream));
        stream->Close();
        Console::WriteLine("deserialized Foo1 from Foo1");
        for each (Foo1kvp kvp in foo1b)
        {
            Console::WriteLine("{0} -> {1}", kvp.Key, kvp.Value);
        }

        // deserialize Foo1 to Foo2
        stream = gcnew FileStream("fooserialized.dat", FileMode::Open, FileAccess::Read);
        Foo2^ foo2 = dynamic_cast<Foo2^>(formatter2->Deserialize(stream));
        stream->Close();
        Console::WriteLine("deserialized Foo2 from Foo1");
        for each (Foo2kvp kvp in foo2)
        {
            Console::WriteLine("{0} -> {1}", kvp.Key, kvp.Value);
        }

        // serialize Foo2
        Foo2^ foo2b = gcnew Foo2;
        foo2b->Add("Two", 7);
        foo2b->Add("Three", 5);
        stream = gcnew FileStream("fooserialized.dat", FileMode::Create, FileAccess::Write);
        formatter2->Serialize(stream, foo2b);
        stream->Close();

        // deserialize Foo2 to Foo2
        stream = gcnew FileStream("fooserialized.dat", FileMode::Open, FileAccess::Read);
        Foo2^ foo2c = dynamic_cast<Foo2^>(formatter2->Deserialize(stream));
        stream->Close();
        Console::WriteLine("deserialized Foo2 from Foo2");
        for each (Foo2kvp kvp in foo2c)
        {
            Console::WriteLine("{0} -> {1}", kvp.Key, kvp.Value);
        }
    }
    catch (Exception^ e)
    {
        Console::WriteLine(e);
        if (stream)
            stream->Close();
    }

    return 0;
}

When running this code, the output is:

Hello World
Deserializing Foo1
deserialized Foo1 from Foo1
2 -> 7
3 -> 5
Deserializing Foo2 from Foo1
deserialized Foo2 from Foo1
2 -> 7
3 -> 5
Deserializing Foo2
deserialized Foo2 from Foo2
Two -> 7
Three -> 5

Regrettably, the same does not work if the class inherits from a List, because List<T> does not implement ISerializable, so the __super::GetObjectData call is not available in a class derived from List<T>. The following code shows how I got it to work for a List in a small application.

using namespace System;
using namespace System::IO;
using namespace System::Collections::Generic;
using namespace System::Runtime::Serialization;

[Serializable]
public ref class Foo1: List<int>
{ };

int
OurVersionNumber(SerializationInfo^ info)
{
   // Serialized Foo1 has no VERSION property, but Foo2 does have it.
   // Don't use info->GetInt32("VERSION") in a try-catch statement,
   // because that is *very* slow when corresponding
   // SerializationExceptions are triggered in the debugger.
   SerializationInfoEnumerator^ it = info->GetEnumerator();
   int version = 1;
   while (it->MoveNext())
   {
      if (it->Name == "VERSION")
      {
         version = (Int32) it->Value;
         break;
      }
   }
   return version;
}

[Serializable]
public ref class Foo2: List<String^>, ISerializable
{
public:
  Foo2() { }

  // NOTE: no "override" on this one, because List<T> doesn't provide this method
  virtual void GetObjectData(SerializationInfo^ info, StreamingContext context)
  {
    info->AddValue("VERSION", 2);
    int size = this->Count;
    List<String^>^ list = gcnew List<String^>(this);
    info->AddValue("This", list);
  }
  Foo2(SerializationInfo^ info, StreamingContext context)
  {
    int version = OurVersionNumber(info);
    if (version == 1)
    {
      int size = info->GetInt32("List`1+_size");
      array<int>^ members = (array<int>^) info->GetValue("List`1+_items", array<int>::typeid);
      for each (int value in members)
      {
        if (!size--)
          break; // done; the remaining 'members' slots are empty
        this->Add(value.ToString());
      }
      Console::WriteLine("Deserializing Foo2 from Foo1");
    }
    else
    {
      List<String^>^ list = (List<String^>^) info->GetValue("This", List<String^>::typeid);
      int size = list->Count;
      this->AddRange(list);
      size = this->Count;
      Console::WriteLine("Deserializing Foo2");
    }
  }
};

ref class MyBinder sealed: public SerializationBinder
{
public:
  virtual Type^ BindToType(String^ assemblyName, String^ typeName) override
  {
    if (typeName == "Foo1")
      typeName = "Foo2";
    return Type::GetType(String::Format("{0}, {1}", typeName, assemblyName));
  }
};

int main(array<System::String ^> ^args)
{
  Console::WriteLine(L"Hello World");
  Foo1^ foo1 = gcnew Foo1;
  foo1->Add(2);
  foo1->Add(3);

  IFormatter^ formatter1 = gcnew Formatters::Binary::BinaryFormatter(); // no translation to Foo2
  IFormatter^ formatter2 = gcnew Formatters::Binary::BinaryFormatter();
  formatter2->Binder = gcnew MyBinder; // translate Foo1 to Foo2
  FileStream^ stream;
  try
  {
    // serialize Foo1
    stream = gcnew FileStream("fooserialized.dat", FileMode::Create, FileAccess::Write);
    formatter1->Serialize(stream, foo1);
    stream->Close();

    // deserialize Foo1 to Foo1
    stream = gcnew FileStream("fooserialized.dat", FileMode::Open, FileAccess::Read);
    Foo1^ foo1b = (Foo1^) formatter1->Deserialize(stream);
    stream->Close();
    Console::WriteLine("deserialized Foo1 from Foo1");
    for each (int value in foo1b)
    {
      Console::WriteLine(value);
    }

    // deserialize Foo1 to Foo2
    stream = gcnew FileStream("fooserialized.dat", FileMode::Open, FileAccess::Read);
    Foo2^ foo2 = (Foo2^) formatter2->Deserialize(stream);
    stream->Close();
    Console::WriteLine("deserialized Foo2 from Foo1");
    for each (String^ value in foo2)
    {
      Console::WriteLine(value);
    }

    // serialize Foo2
    Foo2^ foo2b = gcnew Foo2;
    foo2b->Add("Two");
    foo2b->Add("Three");
    stream = gcnew FileStream("fooserialized.dat", FileMode::Create, FileAccess::Write);
    formatter2->Serialize(stream, foo2b);
    stream->Close();

    // deserialize Foo2 to Foo2
    stream = gcnew FileStream("fooserialized.dat", FileMode::Open, FileAccess::Read);
    Foo2^ foo2c = (Foo2^) formatter2->Deserialize(stream);
    int size = foo2c->Count;
    stream->Close();
    Console::WriteLine("deserialized Foo2 from Foo2");
    for each (String^ value in foo2c)
    {
      Console::WriteLine(value);
    }
  }
  catch (Exception^ e)
  {
    Console::WriteLine(e);
    if (stream)
      stream->Close();
  }

  return 0;
}

This application generates the following output:

Hello World
deserialized Foo1 from Foo1
2
3
Deserializing Foo2 from Foo1
deserialized Foo2 from Foo1
2
3
Deserializing Foo2
deserialized Foo2 from Foo2
Two
Three

However, when using similar code in a very large application to deserialize old, deeply nested data, I keep running into SerializationExceptions with additional information limited to "The object with ID number was referenced in a fixup but does not exist.", and the reported small number is meaningless to me. Inspection of the type names processed by the SerializationBinder shows

System.Collections.Generic.KeyValuePair`2

as well as

System.Collections.Generic.List`1

so the number after the backtick is not fixed. How is that number determined? Can I be sure that it won't suddenly change for a given class if I add other classes to the mix?

Upvotes: 1

How to handle deserialization backward compatibility for changed classes based on generic collections?

Answers (1)

Related Questions