JSON tree modifications that don't break "$ref" references

Question

Question relates to JSON which is result of serialization with PreserveReferencesHandling set to PreserveReferencesHandling.Objects. I'm looking for clever way for modifying a branch of JSON tree (removing or replacing branch in particular) so that references handling is not broken.

Consider following JSON:

{
  "OuterGroup": {
    "ElementA": {
      "$id": "1",
      "data": "A"
    },
    "ElementB": {
      "$id": "2",
      "data": "B"
    }
  },
  "OuterElement": {
    "$ref": "1"
  }
}

I need to replace or remove ElementA. If I do it by JToken.Replace, OuterElement reference will be broken.

First solution that came to my mind was as simple as traversing tree and replacing references with referenced part before any modifications. I am looking for more elegant approach.

Problem background:

Some data in system I work on is persisted in JSON. For some reason I have to migrate it without deserialization (old model is unavailable).

Answer 1

Attempting to modify the JSON directly is going to problematic, especially if your JSON is deeply nested or has multiple references to the same object, so I would not try that approach.

I think best way to do this is to actually deserialize the JSON to an object hierarchy (preserving the references of course), then modify the objects as needed, and finally serialize the hierarchy back to JSON using the PreserveReferencesHandling.Objects setting.

If you had the original object model this would be very easy, as Json.Net supports this functionality already. However, since you don't have the original object model, you'll have to deserialize the JSON into something generic. Normally I would say that using JTokens would be perfect for this, but it turns out that JToken does not support preserving object references.

So, it looks like the only real option is to handle the deserialization manually. Fortunately, this is not as bad as it sounds. You can use a JsonTextReader to read the JSON while recursively building up a hierarchy of generic dictionaries and lists to hold the data. During the process, you can keep track of object references using another dictionary as a lookup table.

Below is a method which encapsulates this logic. Note this method does make some assumptions:

• Your JSON is well-formed (of course)
• For JSON objects that have a $id, that $id is the first property in the object and represents a unique ID relative to all other objects in the JSON
• For JSON objects that have a $ref, the $ref is the only property in that object and it refers to an $id that already appeared earlier in the JSON

All of these should hold true if Json.Net was used to create the JSON in the first place using the PreserveReferencesHandling.Objects setting.

public static object DeserializePreservingReferences(string json)
{
    using (JsonTextReader reader = new JsonTextReader(new StringReader(json)))
    {
        return DeserializePreservingReferences(reader, 
                   new Dictionary<string, Dictionary<string, object>>());
    }
}

private static object DeserializePreservingReferences(JsonTextReader reader,
                         Dictionary<string, Dictionary<string, object>> lookup)
{
    if (reader.TokenType == JsonToken.None)
    {
        reader.Read();
    }

    if (reader.TokenType == JsonToken.StartArray)
    {
        List<object> list = new List<object>();
        while (reader.Read() && reader.TokenType != JsonToken.EndArray)
        {
            list.Add(DeserializePreservingReferences(reader, lookup));
        }
        return list;
    }

    if (reader.TokenType == JsonToken.StartObject)
    {
        Dictionary<string, object> dict = new Dictionary<string, object>();
        while (reader.Read() && reader.TokenType != JsonToken.EndObject)
        {
            string propName = (string)reader.Value;
            reader.Read();

            if (propName == "$ref")
            {
                dict = lookup[reader.Value.ToString()];
            }
            else if (propName == "$id")
            {
                lookup[reader.Value.ToString()] = dict;
            }
            else
            {
                dict.Add(propName, DeserializePreservingReferences(reader, lookup));
            }
        }
        return dict;
    }

    return new JValue(reader.Value).Value;
}

Armed with this method, you can accomplish what you originally wanted. Here is a short demo:

class Program
{
    static void Main(string[] args)
    {
        string json = @"
        {
            ""OuterGroup"": {
                ""ElementA"": {
                    ""$id"": ""1"",
                    ""data"": ""A""
                },
                ""ElementB"": {
                    ""$id"": ""2"",
                    ""data"": ""B""
                }
            },
            ""OuterElement"": {
                ""$ref"": ""1""
            }
        }";

        var root = (Dictionary<string, object>)DeserializePreservingReferences(json);
        var g = (Dictionary<string, object>)root["OuterGroup"];
        g.Remove("ElementA");

        JsonSerializerSettings settings = new JsonSerializerSettings();
        settings.PreserveReferencesHandling = PreserveReferencesHandling.Objects;
        settings.Formatting = Formatting.Indented;
        Console.WriteLine(JsonConvert.SerializeObject(root, settings));
    }
}

Below is the output from the demo program. Although the references may have different $id values in the new JSON, the references have been preserved. Notice the data from the removed ElementA has moved to the OuterElement as you wanted.

{
  "$id": "1",
  "OuterGroup": {
    "$id": "2",
    "ElementB": {
      "$id": "3",
      "data": "B"
    }
  },
  "OuterElement": {
    "$id": "4",
    "data": "A"
  }
}