How can I use Json.NET to deserialize objects by reference, when using a custom format for the reference?

Question

I have JSON in the following format:

{
    "users": [
        {
            "first_name": "John",
            "last_name": "Smith",
            "vet": [ "FOO", "VET-1" ],
            "animals": [ [ "FOO", "ANIMAL-22" ] ]
        },
        {
            "first_name": "Susan",
            "last_name": "Smith",
            "vet": [ "FOO", "VET-1" ]
        }
    ],
    "BAR": {
        "VET-1": {
            "vet_name": "Acme, Inc",
            "vet_id": 456
        },
        "ANIMAL-22": {
            "animal_name": "Fido",
            "species": "dog",
            "animal_id": 789,
            "vet": [ "FOO", "VET-1" ]
        }
    }
}

Some nested objects, or objects referenced more than once, are serialized as references.

The referenced objects are then included in the BAR array at the end of the JSON object, and identified in place by the [ "FOO", "ANIMAL-22" ] array.

(Both FOO and BAR are static constants, and the ANIMAL-22/VET-1 identifiers are semi-random)

Unfortunately, this doesn't match how Json.NET already serializes/deserializes referenced objects and the IReferenceResolver I could implement doesn't seem to allow me to adjust the behaviour enough (it's fixed to use "$ref" for a start).

I've also tried writing a custom JsonConverter for the properties affected, but I can't seem to get a reference to the BAR property of the root object.

Is there any way I can override Json.NET to deserialize the JSON above into this kind of C# class structure?

public class User
{
    [JsonProperty("first_name")]
    public string FirstName { get; set; }

    [JsonProperty("last_name")]
    public string LastName { get; set; }

    [JsonProperty("vet")]
    public Vet Vet { get; set; }

    [JsonProperty("animals")]
    public List<Animal> Animals { get; set; }
}

public class Vet
{
    [JsonProperty("vet_id")]
    public int Id { get; set; }

    [JsonProperty("vet_name")]
    public string Name { get; set; }
}

public class Animal
{
    [JsonProperty("animal_id")]
    public int Id { get; set; }

    [JsonProperty("animal_name")]
    public string Name { get; set; }

    [JsonProperty("vet")]
    public Vet Vet { get; set; }

    [JsonProperty("species")]
    public string Species { get; set; }
}

---

Edit #1: Although I give only Animal and Vet in my example, there are a large number of types referenced in this way and I think I need a 'generic' or type-agnostic solution that would handle any such occurrence of the array structure [ "FOO", "..." ] without needing to code for each C# type individually.

Answer 1

As @dbc said in the comments, there is not an easy way to make Json.Net automatically handle your custom reference format. That said, you can use LINQ-to-JSON (JObjects) to parse the JSON, and with the help of a JsonConverter and a couple of dictionaries, resolve the references and populate your classes while still leaving most of the heavy lifting to Json.Net. Here's the approach I would take:

1. Create a custom generic JsonConverter which can decode the [ "FOO", "<key>" ] reference format and return the corresponding object from a provided dictionary. Here is the code for the converter:

   public class ReferenceConverter<T> : JsonConverter
   {
       private Dictionary<string, T> ReferenceDict { get; set; }
   
       public ReferenceConverter(Dictionary<string, T> referenceDict)
       {
           ReferenceDict = referenceDict;
       }
   
       public override bool CanConvert(Type objectType)
       {
           return objectType == typeof(T);
       }
   
       public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
       {
           JArray array = JArray.Load(reader);
           if (array.Count == 2 && 
               array[0].Type == JTokenType.String && 
               (string)array[0] == "FOO" && 
               array[1].Type == JTokenType.String)
           {
               string key = (string)array[1];
               T obj;
               if (ReferenceDict.TryGetValue(key, out obj))
                   return obj;
   
               throw new JsonSerializationException("No " + typeof(T).Name + " was found with the key \"" + key + "\".");
           }
   
           throw new JsonSerializationException("Reference had an invalid format: " + array.ToString());
       }
   
       public override bool CanWrite
       {
           get { return false; }
       }
   
       public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
       {
           throw new NotImplementedException();
       }
   }
   

2. Parse the JSON into a JObject and build a dictionary of Vets from the BAR section of the JSON.

   JObject data = JObject.Parse(json);
   
   Dictionary<string, Vet> vets = data["BAR"]
       .Children<JProperty>()
       .Where(jp => jp.Value["vet_id"] != null)
       .ToDictionary(jp => jp.Name, jp => jp.Value.ToObject<Vet>());
   

3. Build a dictionary of Animals from the BAR section of the JSON, using the ReferenceConverter<T> and the dictionary from step 2 to resolve Vet references.

   JsonSerializer serializer = new JsonSerializer();
   serializer.Converters.Add(new ReferenceConverter<Vet>(vets));
   
   Dictionary<string, Animal> animals = data["BAR"]
       .Children<JProperty>()
       .Where(jp => jp.Value["animal_id"] != null)
       .ToDictionary(jp => jp.Name, jp => jp.Value.ToObject<Animal>(serializer));
   

4. Finally, deserialize the users list from the JSON, again using the ReferenceConverter<T> plus the two dictionaries (so actually two converter instances now, one per dictionary) to resolve all the references.

   serializer.Converters.Add(new ReferenceConverter<Animal>(animals));
   List<User> users = data["users"].ToObject<List<User>>(serializer);
   

Full demo here: https://dotnetfiddle.net/uUuy7v