CODEWITHSUNDEEP
jsonmongodbgo
Kin
Kin

Reputation: 23

Unmarshal JSON preserving null values

My scenario is the follow:

I have a server/worker made with Go. In a background routine, the server receives messages in JSON format, and then updates a MongoDB database with this data.

One of the problems is, some of the MongoDB data types, such as ObjectId and Date, are often converted to string when they must be represented as JSON, so before inserting that data into the database, I unmarshall that JSON in a structure, and then send that structure to the MongoDB driver. The structure implements methods such as UnmarshalJSON and MarshalBSONValue, so their data types are preserved.

Great, everything is solved. But by using structures I get another problem, supposing I have the following structure:

type Integers struct {
    Foo *int `json:"foo" bson:"foo"`
    Bar *int `json:"bar" bson:"foo"`
    Baz *int `json:"baz" bson:"foo"`
}

And then i receive the following JSON:

{"foo": 0, "bar": null}

With this JSON, I should be updating my database with foo = 0, bar = null, and ignore baz. However, if I unmarshall this JSON in my structure, I'll have the equivalent of:

Integers{
    Foo: 1,
    Bar: nil,
    Baz: nil,
}

But with this I can't tell if I received bar and baz, or they just defaulted to nil, so I can't properly update the database.

How I believe it could be solved:

By having the following structure:

type Integers struct {
    Foo SmartassInt `json:"foo,omitempty" bson:"foo,omitempty"`
    Bar SmartassInt `json:"bar,omitempty" bson:"bar,omitempty"`
    Baz SmartassInt `json:"baz,omitempty" bson:"baz,omitempty"`
}

I would be able to differentiate between a null, and a non-received value, as the following example:

var foo int = 0
var fooPointer *int = &foo
var barPointer *int = nil

integers := Integers{
    Foo: &fooPointer,
    Bar: &barPointer,
    Baz: nil,
}

With this structure, baz will not be inserted in the database, as its value is nil, and nil is ignored thanks to the flag omitempty. bar however is not nil, but it points to nil, which is different from being empty, so it's properly inserted as null in the database.

But how can I achieve this initialization with with the received JSON?

The standard JSON unmarshaller would initialize both bar and baz as nil.

Implementing my own marshaller methods, such as

type NullableInt **int

func (i NullableInt) MarshalJSON() ([]byte, error) {

}

func (i NullableInt) UnmarshalJSON(data []byte) error {

}

Is not possible either, since NullableInt is a pointer, and I can't implement methods on pointers.

So, which approach could I use to solve this problem?

Upvotes: 2

Views: 1438

Answers (1)

torek
torek

Reputation: 488183

On the decode side, you can write a custom unmarshaler for a custom type:

type MaybeInt struct {
    Present bool
    Null    bool
    Value   int64
}

func (m *MaybeInt) UnmarshalJSON(data []byte) error {
    s := string(data)
    m.Present = true
    if s == "null" {
        m.Null = true
        return nil
    }
    v, err := strconv.ParseInt(s, 10, 64)
    m.Value = v
    return err
}

Complete example here. Unfortunately, this doesn't work on the encode side: there is no way for the MarshalJSON handler to indicate that the field is empty. The obvious way would be to return nil, nil from a Marshaler, but that doesn't work. Neither does returning []byte{}, nil.

You might suppose: well, let's use a pointer, and set it to nil when we want to say that the field should be omitted. This works on the decode side, but now the encode side fails, because the encoder sees the literal null and doesn't call our encoder at all!

Ultimately, we can combine both techniques: read into MaybeInt, encode (write) from *MaybeInt. We'll need parallel struct types. We can set the output type based on the input type. I don't claim this to be pretty, and the reflect code in it is terrible (you can also see all my debug tracery), but this actually seems to work: Playground link. In practice, instead of using reflect, you might just write a function for each case of a "maybe" value.

Upvotes: 1

Related Questions