How to deal with mapping intentional NULL values passed through JSON?

Question

Summary

Let's say I am designing an API to do a SQL Server SELECT Query. I have a couple required parameters and some optional parameters. However, if a null value is sent in the payload, this is correct and intentional, but I am unable to tell the difference in the current way I am deserializing the JSON. The value of the property I am deserializing to will be null by default. My issue is that I am unable to tell if it even got filled out or not as there is no marker for this.

Example

In the current way that I am doing this(example below), I am unable to differentiate if a user wants to:

1. Actually look for a null value
2. Remove the null value from serialization if the user is not interested looking up the particular field

using Newtonsoft.Json;

namespace test
{
    public class SomeClass
    {
        public string RequiredProperty {get;set;}
        public string RequiredProperty2 {get;set;}
        public string OptionalProperty3 {get;set;}

        public SomeClass(){}

    }

    class Program
    {
        static void Main(string[] args)
        {   
            // Example JSON payloads
            string JsonExample1 = @"
            {
                ""RequiredProperty"":""search"",
                ""RequiredProperty2"":""this"",
                ""OptionalProperty3"":null
            }";

            string JsonExample2 = @"
            {
                ""RequiredProperty"":""search"",
                ""RequiredProperty2"":""this""
            }";
            
            // Deserializing JsonExample1
            SomeClass sc1 = JsonConvert.DeserializeObject<SomeClass>(JsonExample1);

            // Deserializing JsonExample2 - identical to Example1 even though the INTENTION is completely different
            SomeClass sc2 = JsonConvert.DeserializeObject<SomeClass>(JsonExample2);

            // Now, using the model, I am unable to tell what the user's intentions actually were.
        }
    }
}   

Questions

1. Is my approach to this problem even correct?

• I attempted to use Attributes to create and "IsSet" tag, but found out that was useless as it is attached to the type and cannot be changed at runtime.
• Should I just leave the default value of the json as some kind of special string (i.e. "##notset##")?
• Is there a best practice for this sort of thing? Any examples you guys can think of?

2. Is even proper to have optional parameters passed through JSON? I opted to use JSON payloads for everything because the company I work at has these insane payloads that need to be passed.

Answer 1

The solution I came up with. I had a lot of issues with using JsonConverter so I opted to use the ContractResolver. Seems like this is just generally a limitation with how we currently do our APIs. there are actually a lot of discussions about this. Theres really no way to differentiate something that is its default values and something that has been set. I just opted to create some string filler value to indicate that the user had sent the value in the JSON. This keeps the JSON the EXACT same as it was sent in as it passed along. However, what is mapped end ups entire different and a developer can tell that something hasn't actually been set now.

Unfortunately, I had to come up with 2 contract resolvers because they don't inherently deal with the instance of an object, and I was interested in conditional serialization of the current instance. I ended up having to instantiate a my Custom contract resolver and pass over an instance of the object I was interested in.

If anyone knows how to this better, let me know! Overall, I spent a lot of time looking through GitHub and piecing together different stack overflow posts.

Code: Deserializer

// PreInstalled Packages
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;

// From NuGet - Default
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;

    public class CrudDeserializer: DefaultContractResolver
    {
        // Designated respresentation for a null value passed through JSON, its default is "JsonNull"
        private string NullRepresentation;

        protected override IList<JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
        {
            return type.GetProperties()
                    .Select(p=>{
                        var jp = base.CreateProperty(p, memberSerialization);
                        jp.ValueProvider = new NullToUniqueStringValueProvider(p, this.NullRepresentation);
                        return jp;
                    }).ToList();
        }

        public CrudDeserializer(string nullRepresentation = "JsonNull")
        {
            if(nullRepresentation == null)
            {
                throw new Exception ("nullRepresentation cannot be NULL. It kind of defeats the purpose");
            }

            this.NullRepresentation = nullRepresentation;
        }
    }


// Second class
    public class NullToUniqueStringValueProvider : IValueProvider
    {
        private PropertyInfo MemberInfo;
        private string NullRepresentation;

        public NullToUniqueStringValueProvider(PropertyInfo memberInfo, string nullRepresentation)
        {
            this.MemberInfo = memberInfo;
            this.NullRepresentation = nullRepresentation;
        }

        public object GetValue(object target)
        {
            throw new Exception("This class is not used for serialization");
        }

        public void SetValue(object target, object value)
        {
            if ((string)value == null)
            {
                MemberInfo.SetValue(target, this.NullRepresentation);
            }
            else
            {
                MemberInfo.SetValue(target, value);
            }
        }
    }

Serializer

// PreInstalled Packages
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;

// From NuGet - Default
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;

    public class CrudSerializer<T>: DefaultContractResolver
    {
        private string NullRepresentation;
        private T InstantiatedObject;


        protected override IList<JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
        {
            return type.GetProperties()
                    .Select(p=>{
                        var jp = this.CreateProperty(p, memberSerialization);
                        jp.ValueProvider = new UniqueStringToNull(p, this.NullRepresentation);
                        return jp;
                    }).ToList();
        }

        protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
        {
            JsonProperty property = base.CreateProperty(member, memberSerialization);
            PropertyInfo pi = member as PropertyInfo;
            string value =  (string)pi.GetValue(this.InstantiatedObject);
            if (value == null)
            {
                property.ShouldSerialize =
                    instance =>
                    {
                        return false;
                    };
            }

            return property;
    }

        public CrudSerializer(T someInstance, string nullRepresentation = "JsonNull")
        {
            if(nullRepresentation == null)
            {
                throw new Exception ("nullRepresentation cannot be NULL. It kind of defeats the purpose");
            }

            this.NullRepresentation = nullRepresentation;
            this.InstantiatedObject = someInstance;
        }
    }

    public class UniqueStringToNull : IValueProvider
    {
        private PropertyInfo MemberInfo;
        private string NullRepresentation;

        public UniqueStringToNull(PropertyInfo memberInfo, string nullRepresentation)
        {
            this.MemberInfo = memberInfo;
            this.NullRepresentation = nullRepresentation;
        }

        public object GetValue(object target)
        {
            object result =  MemberInfo.GetValue(target);
            if (MemberInfo.PropertyType == typeof(string) && (string)result == this.NullRepresentation)
            {
                result = null;
            }
            return result;

        }

        public void SetValue(object target, object value)
        {
            throw new Exception ("This ContractResolver cannot be used for deserialization");
        }
    }

Example:

    public class SomeClass
    {
        
        public string RequiredProperty {get;set;}
        public string RequiredProperty2 {get;set;}

        public string OptionalProperty3 {get;set;}
        public string OptionalProperty4 {get;set;}

        public SomeClass(){}

    }
    class Program
    {
        static void Main(string[] args)
        {   
            // Example JSON payloads
            string JsonExample1 = @"
            {""RequiredProperty"":""search"",""RequiredProperty2"":""this"",""OptionalProperty3"": null}";

            string JsonExample2 = @"
            {""RequiredProperty"":""search"",""RequiredProperty2"":""this""}";
            
            JsonSerializerSettings deserializationSettings = new JsonSerializerSettings { 
                ContractResolver = new CrudDeserializer()
                };
                

            // Deserializing/Serializing JsonExample1
            SomeClass sc1 = JsonConvert.DeserializeObject<SomeClass>(JsonExample1, deserializationSettings );

            // Passing over current instance of object to ContractResolver
            JsonSerializerSettings serializationSettings1 = new JsonSerializerSettings { 
                ContractResolver = new CrudSerializer<SomeClass>(sc1)
                };
            string json1 =  JsonConvert.SerializeObject(sc1, Formatting.None, serializationSettings1);


            // Deserializing/Serializing JsonExample2 
            SomeClass sc2 = JsonConvert.DeserializeObject<SomeClass>(JsonExample2, deserializationSettings);

            // Passing over current instance of object to ContractResolver
            JsonSerializerSettings serializationSettings2 = new JsonSerializerSettings { 
                ContractResolver = new CrudSerializer<SomeClass>(sc2)
                };

            string json2 =  JsonConvert.SerializeObject(sc2,  Formatting.None, serializationSettings2);

            // Done, JSON is the exact same no matter how many times I deserialize.
            // However, in the background, I am able to tell now if a NULL value was sent!
        }

    }

Answer 2

The comment provided by Andy is correct, I also don't know the difference between the two you mentioned in your question. I think the value of OptionalProperty3 is null is same with OptionalProperty3 is not-set.

If you still want to distinguish between them, here I can provide a workaround for your reference. Replace the null in JsonExample1 with "null" string, please refer to my code below: