JObject.ToBsonDocument dropping values

Question

I'm inserting raw JSON into a collection and finding that what is stored in the database is missing the values. For example, my collection is a collection of BsonDocuments:

_products = database.GetCollection<BsonDocument>("products");

The code to insert the JSON into the collection:

public int AddProductDetails(JObject json)
{
    var doc = json.ToBsonDocument(DictionarySerializationOptions.Document);
    _products.Insert(doc);
}

The JSON that is passed in looks like this:

{
  "Id": 1,
  "Tags": [
    "book",
    "database"
  ],
  "Name": "Book Name",
  "Price": 12.12
}

But, what is persisted in the collection is just the properties with no values.

{
  "_id": {
    "$oid": "5165c7e10fdb8c09f446d720"
  },
  "Id": [],
  "Tags": [
    [],
    []
  ],
  "Name": [],
  "Price": []
}

Why are the values being dropped?

Answer 1

Most of the answers here involve serializing to and then deserializing from a string. Here is a solution that serializes to/from raw BSON instead. It requires the Newtonsoft.Json.Bson nuget package.

using System.IO;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Serializers;
using Newtonsoft.Json;
using Newtonsoft.Json.Bson;
using Newtonsoft.Json.Linq;

namespace Zonal.EventPublisher.Worker
{
    public class JObjectSerializer : SerializerBase<JObject>
    {
        public override JObject Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
        {
            using (var stream = new MongoDB.Bson.IO.ByteBufferStream(context.Reader.ReadRawBsonDocument()))
            using (JsonReader reader = new BsonDataReader(stream))
            {
                return JObject.Load(reader);
            }
        }

        public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, JObject value)
        {
            using (var stream = new MemoryStream())
            using (JsonWriter writer = new BsonDataWriter(stream))
            {
                value.WriteTo(writer);

                var buffer = new MongoDB.Bson.IO.ByteArrayBuffer(stream.ToArray());
                context.Writer.WriteRawBsonDocument(buffer);
            }
        }
    }
}

Don't forget to register the serializer with:

BsonSerializer.RegisterSerializer(new JObjectSerializer());

After that you can convert your JObject to a BsonDocument by using the MongoDB.Bson.BsonExtensionMethods.ToBsonDocument extension method:

var myBsonDocument = myJObject.ToBsonDocument()

And convert a BsonDocument back to a JObject by using the MongoDB.Bson.Serialization.BsonSerializer class:

var myJObject = BsonSerializer.Deserialize<JObject>(myBsonDocument);

Answer 2

I use the following. It's based on Simon's answer, thanks for the idea, and works in the same way, avoiding unnecessary serialization / deserialization into string.

It's just a bit more compact, thanks to Linq and C# 10:

public static BsonDocument ToBsonDocument(this JObject o) =>
    new(o.Properties().Select(p => new BsonElement(p.Name, p.Value.ToBsonValue())));

public static BsonValue ToBsonValue(this JToken t) =>
    t switch
    {
        JObject o => o.ToBsonDocument(),
        JArray a => new BsonArray(a.Select(ToBsonValue)),
        JValue v => BsonValue.Create(v.Value),
        _ => throw new NotSupportedException($"ToBsonValue: {t}")
    };

Answer 3

The problem when using JObject.ToString, BsonDocument.Parse, etc. is the performance is not very good because you do the same operations multiple times, you do string allocations, parsing, etc.

So, I have written a function that converts a JObject to an IEnumerable<KeyValuePair<string, object>> (only using enumerations), which is a type usable by one of the BsonDocument constructors. Here is the code:

public static BsonDocument ToBsonDocument(this JObject jo)
{
    if (jo == null)
        return null;

    return new BsonDocument(ToEnumerableWithObjects(jo));
}

public static IEnumerable<KeyValuePair<string, object>> ToEnumerableWithObjects(this JObject jo)
{
    if (jo == null)
        return Enumerable.Empty<KeyValuePair<string, object>>();

    return new JObjectWrapper(jo);
}

private class JObjectWrapper : IEnumerable<KeyValuePair<string, object>>
{
    private JObject _jo;

    public JObjectWrapper(JObject jo)
    {
        _jo = jo;
    }

    public IEnumerator<KeyValuePair<string, object>> GetEnumerator() => new JObjectWrapperEnumerator(_jo);
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    public static object ToValue(JToken token)
    {
        object value;
        switch (token.Type)
        {
            case JTokenType.Object:
                value = new JObjectWrapper((JObject)token);
                break;

            case JTokenType.Array:
                value = new JArrayWrapper((JArray)token);
                break;

            default:
                if (token is JValue jv)
                {
                    value = ((JValue)token).Value;
                }
                else
                {
                    value = token.ToString();
                }
                break;
        }
        return value;
    }
}

private class JArrayWrapper : IEnumerable
{
    private JArray _ja;

    public JArrayWrapper(JArray ja)
    {
        _ja = ja;
    }

    public IEnumerator GetEnumerator() => new JArrayWrapperEnumerator(_ja);
}

private class JArrayWrapperEnumerator : IEnumerator
{
    private IEnumerator<JToken> _enum;

    public JArrayWrapperEnumerator(JArray ja)
    {
        _enum = ja.GetEnumerator();
    }

    public object Current => JObjectWrapper.ToValue(_enum.Current);
    public bool MoveNext() => _enum.MoveNext();
    public void Reset() => _enum.Reset();
}

private class JObjectWrapperEnumerator : IEnumerator<KeyValuePair<string, object>>
{
    private IEnumerator<KeyValuePair<string, JToken>> _enum;

    public JObjectWrapperEnumerator(JObject jo)
    {
        _enum = jo.GetEnumerator();
    }

    public KeyValuePair<string, object> Current => new KeyValuePair<string, object>(_enum.Current.Key, JObjectWrapper.ToValue(_enum.Current.Value));
    public bool MoveNext() => _enum.MoveNext();
    public void Dispose() => _enum.Dispose();
    public void Reset() => _enum.Reset();
    object IEnumerator.Current => Current;
}

Answer 4

Here is an updated version of Andrew DeVries's answer that includes handling for serializing/deserializing null values.

public class JObjectSerializer : SerializerBase<JObject>
{
    public override JObject Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
    {
        if (context.Reader.CurrentBsonType != BsonType.Null)
        {
            var myBSONDoc = BsonDocumentSerializer.Instance.Deserialize(context);
            return JObject.Parse(myBSONDoc.ToStrictJson());
        }
        else
        {
            context.Reader.ReadNull();
            return null;
        }
    }

    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, JObject value)
    {
        if (value != null)
        {
            var myBSONDoc = BsonDocument.Parse(value.ToString());
            BsonDocumentSerializer.Instance.Serialize(context, myBSONDoc);
        }
        else
        {
            context.Writer.WriteNull();
        }
    }
}

The ToStrictJson() call is an extension method that wraps the call to the built-in BSON ToJson() method to include setting the output mode to strict. If this is not done, the parsing will fail because BSON type constructors will remain in the JSON output (ObjectId(), for example).

Here is the implementation of ToStrictJson() as well:

public static class MongoExtensionMethods
{
    /// <summary>
    /// Create a JsonWriterSettings object to use when serializing BSON docs to JSON.
    /// This will force the serializer to create valid ("strict") JSON.
    /// Without this, Object IDs and Dates are ouput as {"_id": ObjectId(ds8f7s9d87f89sd9f8d9f7sd9f9s8d)}
    ///  and {"date": ISODate("2020-04-14 14:30:00:000")} respectively, which is not valid JSON
    /// </summary>
    private static JsonWriterSettings jsonWriterSettings = new JsonWriterSettings()
    {
        OutputMode = JsonOutputMode.Strict
    };

    /// <summary>
    /// Custom extension method to convert MongoDB objects to JSON using the OutputMode = Strict setting.
    /// This ensure that the resulting string is valid JSON.
    /// </summary>
    /// <typeparam name="TNominalType">The type of object to convert to JSON</typeparam>
    /// <param name="obj">The object to conver to JSON</param>
    /// <returns>A strict JSON string representation of obj.</returns>
    public static string ToStrictJson<TNominalType>(this TNominalType obj)
    {
        return BsonExtensionMethods.ToJson<TNominalType>(obj, jsonWriterSettings);
    }
}

Answer 5

I ran into this issue when I had a C# class with a property of type JObject.

My Solution was to create JObjectSerializer for MondoDB and add the attribute to the property so Mongo serializer uses it. I assume if I tried hard enough I could register the below serializer in Mongo as the global one for this type as well.

Register serializer for property processing:

[BsonSerializer(typeof(JObjectSerializer))]
public JObject AdditionalData { get; set; }

The serializer itself:

public class JObjectSerializer : SerializerBase<JObject> // IBsonSerializer<JObject>
{
    public override JObject Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
    {
        var myBSONDoc = BsonDocumentSerializer.Instance.Deserialize(context);
        return JObject.Parse(myBSONDoc.ToString());
    }

    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, JObject value)
    {
        var myBSONDoc = MongoDB.Bson.BsonDocument.Parse(value.ToString());
        BsonDocumentSerializer.Instance.Serialize(context, myBSONDoc);
    }
}

Answer 6

Have you tried using the BsonSerializer?

using MongoDB.Bson.Serialization;
[...]
var document = BsonSerializer.Deserialize<BsonDocument>(json);

BsonSerializer works with strings, so if the JSON argument is a JObject(or JArray, JRaw etc) you have to serialize it with JsonConvert.SerializeObject()

Answer 7

This does what I was expecting.

    public int AddProductDetails(JObject json)
    {
        BsonDocument doc = BsonDocument.Parse(json.ToString());
        _products.Insert(doc);
    }