How to avoid data loss on ScriptableObject data structure changes

Question

I am working with ScriptableObject assets to store my data for my multilanguage app project. This works pretty well, except for the case when I want to change the existing data structure.

In my case, the data structure looks like this:

[CreateAssetMenu(fileName = "Data", menuName = "ScriptableObjects/I18NData", order = 1)]
public class I18NData : ScriptableObject
{
    public I18NSpriteData[] Sprites;
    public I18NTextData[] Texts;
}

[System.Serializable]
public class I18NSpriteData
{
    public string Label;
    public Sprite SpriteEN;
    public Sprite SpriteFR;
    public Sprite SpriteSG;
    public Sprite SpriteES;
    public Sprite SpriteDE;
    public Sprite SpriteIT;
}

[System.Serializable]
public class I18NTextData
{
    public string Label;

    [TextArea]
    public string TextEN;

    [TextArea]
    public string TextFR;

    [TextArea]
    public string TextSG;

    [TextArea]
    public string TextES;

    [TextArea]
    public string TextDE;

    [TextArea]
    public string TextIT;
}

Now I add data in the Unity Editor and all is working and fine. But as soon as I change something on the existing data structure, all data in the Editor is lost. The ScriptableObject seems to be reset to empty state.

This is especially annoying as we are in the middle of development and data structure changes are unavoidable at this stage...

What do you do in order to avoid such situations? Do you have some scripts that generate ScriptableObjects based on other files, such as JSON? Or is there a simple way from within Unity?

Thanks for any hint!

Answer 1

If you want to rename a serialized field, but keep the data, Unity has an attribute called FormerlySerializedAs, which will maintain the old serialized field name.

[CreateAssetMenu(fileName = "Data", menuName = "ScriptableObjects/I18NData", order = 1)]
public class I18NData : ScriptableObject
{

    [FormerlySerializedAs("Sprites")]
    public I18NSpriteData[] Sprites2;
    public I18NTextData[] Texts;
}

Another option is you could make the fields private and use the SerializeField attribute. You could then freely change properties that access or modify the fields over time without affecting serialization:

[CreateAssetMenu(fileName = "Data", menuName = "ScriptableObjects/I18NData", order = 1)]
public class I18NData : ScriptableObject
{
    [SerializeField]
    private I18NSpriteData[] Sprites;
    public I18NSpriteData[] Sprites2 {
            get { return Sprites; }
            set { Sprites = value; }
    }

    [SerializeField]
    private I18NTextData[] Texts;
    public I18NTextData[] Texts2 {
            get { return Texts; }
            set { Texts= value; }
    }
}

Another thing you might want to consider is using the ISerializationCallbackReceiver interface to have more control over Unity's serialization process. It provides two methods, OnBeforeSerialize and OnAfterDeserialize, which are called during Unity's serialization and deserialization process.

This is probably more useful when serializing complex structures that Unity can't normally handle, like Dictionaries, but depending on the types of changes you are making it might be helpful. You could potentially use it for migrating data between revisions.

Answer 2

You are probably changing the data structure in such a way that make Unity unable to unpack the serialized data in a desired way. Hence the missing data.

If the Force Text serialization mode is selected all the ScriptableObject's will be serialized into the human readable format YAML.

When read in a text-editor the ScriptableObject will look something like this:

%YAML 1.1
%TAG !u! tag:unity3d.com,2011:
--- !u!114 &11400000
MonoBehaviour:
  m_ObjectHideFlags: 0
  m_CorrespondingSourceObject: {fileID: 0}
  m_PrefabInternal: {fileID: 0}
  m_GameObject: {fileID: 0}
  m_Enabled: 1
  m_EditorHideFlags: 0
  m_Script: {fileID: 11500000, guid: 9228bfb9e457c5341920079380c382ba, type: 3}
  m_Name: Data
  m_EditorClassIdentifier:
  Sprites:
  - Label:
    SpriteEN: {fileID: 10913, guid: 0000000000000000f000000000000000, type: 0}
    SpriteFR: {fileID: 10907, guid: 0000000000000000f000000000000000, type: 0}
    SpriteSG: {fileID: 10915, guid: 0000000000000000f000000000000000, type: 0}
    SpriteES: {fileID: 10911, guid: 0000000000000000f000000000000000, type: 0}
    SpriteDE: {fileID: 10913, guid: 0000000000000000f000000000000000, type: 0}
    SpriteIT: {fileID: 10905, guid: 0000000000000000f000000000000000, type: 0}
  Texts:
  - Label:
    TextEN: "Hello\t"
    TextFR: Salut!
    TextSG: ????
    TextES: Holla!
    TextDE: Bratwurst
    TextIT: Pizza!

With some grasp of YAML you could change the data to fit the new data structure manually.

Answer 3

Some changes to the structure will always cause the data loss, therefore you should not populate it with data before you are sure about the final layout of your data, at least the main staff, unity serialization should support adding new fields though, but in any case, I wouldn't rely on it.

---

There are a couple of technics you may use to improve your workflow.

1. Create a hierarchy of data assets. For example, make your single translation instances as a separate scriptable object to encapsulate your structs (btw, use structs instead of classes for the value objects like "I18NSpriteData" or "I18NTextData"), then link all assets inside "I18NData".
2. Use external data store, like JSON or a database. This implies to develop an import/export tool for the Editor, but in many cases it worth doing so, especially when you are thinking to implement dynamic translations in the future (by downloading from a server, for example). You can also think about a more extensible approach for the languages and instead of hardcoding them as a struct field, there may be an array of translations for each element, for example:

[System.Serializable]
public struct I18NTextData
{
    public string Label;
    public I18NTextDataTranslation[] translations;
}
[System.Serializable]
public struct I18NTextDataTranslation
{
    public string lang;
    public string content;
}