Json.Net: How to ignore null elements in array deserializing a JSON

Question

I have this JSON:

{
    "Variable1": "1",
    "Variable2": "50000",
    "ArrayObject": [null]
}

I have this stubs:

public class Class1
{
  public string Variable1 { get; set; }
  public string Variable2 { get; set; }
  [JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
  public List<ArrayObject> ArrayObject { get; set; }
}

public class ArrayObject
{
  public string VariableArray1 { get; set; }
  public string VariableArray2 { get; set; }
}

I'd like to ignore the null elements inside array preferably using the json settings or some sort of converter. So the result should be an empty array in that case or null.

Here is the code I've been trying to make this work.

class Program
{
  static void Main(string[] args)
  {
    string json = @"{
      ""Variable1"": ""1"",
      ""Variable2"": ""50000"",
      ""ArrayObject"": [null]
    }";

    var settings = new JsonSerializerSettings()
    {
      ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
      NullValueHandling = NullValueHandling.Ignore,
    };

    Class1 class1 = JsonConvert.DeserializeObject<Class1>(json, settings);

    Console.WriteLine(class1.ArrayObject == null);
    Console.WriteLine(class1.ArrayObject.Count());

    foreach (var item in class1.ArrayObject)
    {
      Console.WriteLine(item.VariableArray1);
      Console.WriteLine(item.VariableArray2);
      Console.WriteLine("#######################");
    }
  }

  public class Class1
  {
    public string Variable1 { get; set; }
    public string Variable2 { get; set; }
    [JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
    public List<ArrayObject> ArrayObject { get; set; }
  }

  public class ArrayObject
  {
    public string VariableArray1 { get; set; }
    public string VariableArray2 { get; set; }
  }
}

I thought that using NullValueHandling = NullValueHandling.Ignore would make it work. Apparently not. Any ideas?

Update: I need a global solution, I don't want to have to modify every viewmodel inside my project.

Answer 1

Setting NullValueHandling = NullValueHandling.Ignore will not filter null values from JSON arrays automatically during deserialization because doing so would cause the remaining items in the array to be re-indexed, rendering invalid any array indices that might have been stored elsewhere in the serialization graph.

If you don't care about preserving array indices and want to filter null values from the array during deserialization anyway, you will need to implement a custom JsonConverter such as the following:

public class NullFilteringListConverter<T> : JsonConverter<List<T>>
{
    public override List<T> ReadJson(JsonReader reader, Type objectType, List<T> existingValue, bool hasExistingValue, JsonSerializer serializer)
    {
        if (reader.MoveToContentAndAssert().TokenType == JsonToken.Null)
            return null;
        var list = existingValue as List<T> ?? (List<T>)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
        serializer.Populate(reader, list);
        list.RemoveAll(i => i == null);
        return list;
    }

    public override bool CanWrite => false;
    public override void WriteJson(JsonWriter writer, List<T> value, JsonSerializer serializer) => throw new NotImplementedException();
}

public static partial class JsonExtensions
{
    public static JsonReader MoveToContentAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (reader.TokenType == JsonToken.None)       // Skip past beginning of stream.
            reader.ReadAndAssert();
        while (reader.TokenType == JsonToken.Comment) // Skip past comments.
            reader.ReadAndAssert();
        return reader;
    }

    public static JsonReader ReadAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (!reader.Read())
            throw new JsonReaderException("Unexpected end of JSON stream.");
        return reader;
    }
}

And apply it to your model as follows:

public class Class1
{
    public string Variable1 { get; set; }
    public string Variable2 { get; set; }
    [JsonConverter(typeof(NullFilteringListConverter<ArrayObject>))]
    public List<ArrayObject> ArrayObject { get; set; }
}

Or, add it in settings as follows:

var settings = new JsonSerializerSettings()
{
    ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
    NullValueHandling = NullValueHandling.Ignore,
    Converters = { new NullFilteringListConverter<ArrayObject>() }, 
};

Notes:

• Since you didn't ask about filtering null values during serialization, I didn't implement it, however it would be easy to do by changing CanWrite => true; and replacing WriteJson() with:

   public override void WriteJson(JsonWriter writer, List<T> value, JsonSerializer serializer) => serializer.Serialize(writer, value.Where(i => i != null));
  

Demo fiddles here and here.

Update

I need a global solution. If you need to automatically filter all null values from all possible List<T> objects in every model, the following JsonConverter will do the job:

public class NullFilteringListConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        if (objectType.IsArray || objectType == typeof(string) || objectType.IsPrimitive)
            return false;
        var itemType = objectType.GetListItemType();
        return itemType != null && (!itemType.IsValueType || Nullable.GetUnderlyingType(itemType) != null);
    }

    object ReadJsonGeneric<T>(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var list = existingValue as List<T> ?? (List<T>)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
        serializer.Populate(reader, list);
        list.RemoveAll(i => i == null);
        return list;
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.MoveToContentAndAssert().TokenType == JsonToken.Null)
            return null;
        var itemType = objectType.GetListItemType();
        var method = typeof(NullFilteringListConverter).GetMethod("ReadJsonGeneric", BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Public);
        try
        {
            return method.MakeGenericMethod(new[] { itemType }).Invoke(this, new object[] { reader, objectType, existingValue, serializer });
        }
        catch (Exception ex)
        {
            // Wrap the TargetInvocationException in a JsonSerializerException
            throw new JsonSerializationException("Failed to deserialize " + objectType, ex);
        }
    }

    public override bool CanWrite => false;
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) => throw new NotImplementedException();
}

public static partial class JsonExtensions
{
    internal static Type GetListItemType(this Type type)
    {
        // Quick reject for performance
        if (type.IsPrimitive || type.IsArray || type == typeof(string))
            return null;
        while (type != null)
        {
            if (type.IsGenericType)
            {
                var genType = type.GetGenericTypeDefinition();
                if (genType == typeof(List<>))
                    return type.GetGenericArguments()[0];
            }
            type = type.BaseType;
        }
        return null;
    }
}

And add it to settings as follows:

var settings = new JsonSerializerSettings()
{
    ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
    NullValueHandling = NullValueHandling.Ignore,
    Converters = { new NullFilteringListConverter() },
};

Class1 class1 = JsonConvert.DeserializeObject<Class1>(json, settings);

With this converter, adding [JsonConverter(typeof(NullFilteringListConverter<ArrayObject>))] to ArrayObject is no longer required. Do note that all List<T> instances in your deserialization graph may get re-indexed whenever these settings are used! Make sure you really want this as the side-effects of changing indices of items referred to elsewhere by index may include data corruption (incorrect references) rather than an outright ArgumentOutOfRangeException.

Demo fiddle #3 here.

Answer 2

NullValueHandling.Ignore does not filter null values from an array. But you can do this easily by adding a deserialization callback method in your class to filter out the nulls:

public class Class1
{
    public string Variable1 { get; set; }
    public string Variable2 { get; set; }
    public List<ArrayObject> ArrayObject { get; set; }

    [OnDeserialized]
    internal void OnDeserialized(StreamingContext context)
    {
        ArrayObject?.RemoveAll(o => o == null);
    }
}

Working demo: https://dotnetfiddle.net/v9yn7j

Answer 3

You could also have a custom setter that filters out null values.

private List<ArrayObject> _arrayObject;
public List<ArrayObject> ArrayObject
{
    get => _arrayObject;
    set
    {
        _arrayObject = value.Where(x => x != null).ToList();
    }
}

Fiddle working here https://dotnetfiddle.net/ePp0A2