JsonConvert.DeserializeObject w/ DynamicObject and TypeCreationConverter

Question

I have a class EntityBase that derives from DynamicObject without an empty default constructor.

// this is not the actual type but a mock to test the behavior with
public class EntityBase : DynamicObject
{
    public string EntityName { get; private set; }

    private readonly Dictionary<string, object> values = new Dictionary<string, object>();

    public EntityBase(string entityName)
    {
        this.EntityName = entityName;
    }

    public virtual object this[string fieldname]
    {
        get
        {
            if (this.values.ContainsKey(fieldname))
                return this.values[fieldname];
            return null;
        }
        set
        {
            if (this.values.ContainsKey(fieldname))
                this.values[fieldname] = value;
            else
                this.values.Add(fieldname, value);          
        }
    }

    public override IEnumerable<string> GetDynamicMemberNames()
    {
        return this.values.Keys.ToList();
    }

    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        result = this[binder.Name];
        return true;
    }

    public override bool TrySetMember(SetMemberBinder binder, object value)
    {
        this[binder.Name] = value;
        return true;
    }
}

the JSON I'd like to deserialize looks like this:

{'Name': 'my first story', 'ToldByUserId': 255 }

EntityBase has neither the Name nor the ToldByUserId property. They should be added to the DynamicObject.

If I let DeserializeObject create the object like this everything works as expected:

var story = JsonConvert.DeserializeObject<EntityBase>(JSON);

but since I don't have an empty default constructor and can't change the class I went for a CustomCreationConverter :

public class StoryCreator : CustomCreationConverter<EntityBase>
{
    public override EntityBase Create(Type objectType)
    {
        return new EntityBase("Story");
    }
}

but

var stroy = JsonConvert.DeserializeObject<EntityBase>(JSON, new StoryCreator());

throws

Cannot populate JSON object onto type 'DynamicObjectJson.EntityBase'. Path 'Name', line 1, position 8.

It seems that the DeserializeObject calls PopulateObject on the object that was created by the CustomCreationConverter. When I try to do this manually the error stays the same

JsonConvert.PopulateObject(JSON, new EntityBase("Story"));

I further assume that PopulateObject does not check if the target type derives from DynamicObject and therefore does not fall back to TrySetMember.

Note that I don't have influence on the EntityBase type definition, it's from an external library and cannot be changed.

Any insights would be highly appreciated!

Edit: added an example: https://dotnetfiddle.net/EGOCFU

Answer 1

You seem to have stumbled on a couple of bugs or limitations in Json.NET's support for deserializing dynamic objects (defined as those for which a JsonDynamicContract is generated):

1. Support for parameterized constructors is not present. Even if one is marked with [JsonConstructor] it will not get used.

   Here the necessary logic to pre-load all the properties seems to be entirely missing from JsonSerializerInternalReader.CreateDynamic(). Compare with JsonSerializerInternalReader.CreateNewObject() which indicates what would be required.

   Since the logic looks fairly elaborate this might be a limitation rather than a bug. And actually there is closed issue #47 about this indicating that it's not implemented:

   There would be a fair bit of work to add this feature. You are welcome to submit a pull request if you do add it.

2. Json.NET has no ability to populate a preexisting dynamic object. Unlike for regular objects (those for which a JsonObjectContract is generated), logic for construction and population is contained entirely in the previously-mentioned JsonSerializerInternalReader.CreateDynamic().

   I don't see why this couldn't be implemented with a fairly simple code restructuring. You might submit an issue asking for this. If this were implemented, your StoryCreator would work as-is.

In the absence of either #1 or #2, it's possible to create a custom JsonConverter whose logic is modeled roughly on JsonSerializerInternalReader.CreateDynamic() which calls a specified creation method then populates both dynamic and non-dynamic properties, like so:

public class EntityBaseConverter : ParameterizedDynamicObjectConverterBase<EntityBase>
{
    public override EntityBase CreateObject(JObject jObj, Type objectType, JsonSerializer serializer, ICollection<string> usedParameters)
    {
        var entityName = jObj.GetValue("EntityName", StringComparison.OrdinalIgnoreCase);
        if (entityName != null)
        {
            usedParameters.Add(((JProperty)entityName.Parent).Name);
        }
        var entityNameString = entityName == null ? "" : entityName.ToString();
        if (objectType == typeof(EntityBase))
        {
            return new EntityBase(entityName == null ? "" : entityName.ToString());             
        }
        else
        {
            return (EntityBase)Activator.CreateInstance(objectType, new object [] { entityNameString });
        }           
    }
}

public abstract class ParameterizedDynamicObjectConverterBase<T> : JsonConverter where T : DynamicObject
{
    public override bool CanConvert(Type objectType) { return typeof(T).IsAssignableFrom(objectType); } // Or possibly return objectType == typeof(T);

    public abstract T CreateObject(JObject jObj, Type objectType, JsonSerializer serializer, ICollection<string> usedParameters);

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        // Logic adapted from JsonSerializerInternalReader.CreateDynamic()
        // https://github.com/JamesNK/Newtonsoft.Json/blob/master/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs#L1751
        // By James Newton-King https://github.com/JamesNK

        var contract = (JsonDynamicContract)serializer.ContractResolver.ResolveContract(objectType);

        if (reader.TokenType == JsonToken.Null)
            return null;

        var jObj = JObject.Load(reader);

        var used = new HashSet<string>();
        var obj = CreateObject(jObj, objectType, serializer, used);

        foreach (var jProperty in jObj.Properties())
        {
            var memberName = jProperty.Name;
            if (used.Contains(memberName))
                continue;
            // first attempt to find a settable property, otherwise fall back to a dynamic set without type
            JsonProperty property = contract.Properties.GetClosestMatchProperty(memberName);

            if (property != null && property.Writable && !property.Ignored)
            {
                var propertyValue = jProperty.Value.ToObject(property.PropertyType, serializer);
                property.ValueProvider.SetValue(obj, propertyValue);
            }
            else
            {
                object propertyValue;
                if (jProperty.Value.Type == JTokenType.Null)
                    propertyValue = null;
                else if (jProperty.Value is JValue)
                    // Primitive
                    propertyValue = ((JValue)jProperty.Value).Value;
                else
                    propertyValue = jProperty.Value.ToObject<IDynamicMetaObjectProvider>(serializer);
                // Unfortunately the following is not public!
                // contract.TrySetMember(obj, memberName, propertyValue);
                // So we have to duplicate the logic of what Json.NET has already done.
                CallSiteCache.SetValue(memberName, obj, propertyValue);
            }               
        }
        return obj;
    }

    public override bool CanWrite { get { return false; } }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

internal static class CallSiteCache
{
    // Adapted from the answer to 
    // https://stackoverflow.com/questions/12057516/c-sharp-dynamicobject-dynamic-properties
    // by jbtule, https://stackoverflow.com/users/637783/jbtule
    // And also
    // https://github.com/mgravell/fast-member/blob/master/FastMember/CallSiteCache.cs
    // by Marc Gravell, https://github.com/mgravell

    private static readonly Dictionary<string, CallSite<Func<CallSite, object, object, object>>> setters 
        = new Dictionary<string, CallSite<Func<CallSite, object, object, object>>>();

    public static void SetValue(string propertyName, object target, object value)
    {
        CallSite<Func<CallSite, object, object, object>> site;

        lock (setters)
        {
            if (!setters.TryGetValue(propertyName, out site))
            {
                var binder = Binder.SetMember(CSharpBinderFlags.None,
                       propertyName, typeof(CallSiteCache),
                       new List<CSharpArgumentInfo>{
                               CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, null),
                               CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, null)});
                setters[propertyName] = site = CallSite<Func<CallSite, object, object, object>>.Create(binder);
            }
        }

        site.Target(site, target, value);
    }
}

Then use it like:

var settings = new JsonSerializerSettings
{
    Converters = { new EntityBaseConverter() },
};
var stroy = JsonConvert.DeserializeObject<EntityBase>(JSON, settings);

Since it seems like EntityBase may be a base class for multiple derived classes, I wrote the converter to work for all derived types of EntityBase with the assumption that they all have a parameterized constructor with the same signature.

Note I am taking the EntityName from the JSON. If you would prefer to hardcode it to "Story" you could do that, but you should still add the actual name of the EntityName property to the usedParameters collection to prevent a dynamic property with the same name from getting created.

Sample working .Net fiddle here.