CODEWITHSUNDEEP
mysqlmongodbgomarshallingunmarshalling
DevOverlord
DevOverlord

Reputation: 446

Dealing with NULLs when migrating from MySQL to Mongo in Go

Setup

I started a project using MySQL and as such, my project has some helper types that assist with dealing with nulls, both when unmarshalling incoming data on the API, inputting data into the DB, and then the inverse of that, pulling data out of the Database and responding with said data to the API.

For the purposes of this question, we'll deal with a struct i have that represents a Character.

type Character struct {
    MongoID          primitive.ObjectID `bson:"_id" json:"-"`
    ID               uint64             `bson:"id" json:"id"`
    Name             string             `bson:"name" json:"name"`
    CorporationID    uint               `bson:"corporation_id" json:"corporation_id"`
    AllianceID       null.Uint           `bson:"alliance_id" json:"alliance_id,omitempty"`
    FactionID        null.Uint          `bson:"faction_id" json:"faction_id,omitempty"`
    SecurityStatus   float64            `bson:"security_status" json:"security_status"`
    NotModifiedCount uint               `bson:"not_modified_count" json:"not_modified_count"`
    UpdatePriority   uint               `bson:"update_priority" json:"update_priority"`
    Etag             null.String        `bson:"etag" json:"etag"`
    CachedUntil      time.Time          `bson:"cached_until" json:"cached_until"`
    CreatedAt        time.Time          `bson:"created_at" json:"created_at"`
    UpdatedAt        time.Time          `bson:"updated_at" json:"updated_at"`
}

I want to specifically concentrate on the AllianceID property of type null.Uint which is represented with the following struct:

// Uint is an nullable uint.
type Uint struct {
    Uint  uint
    Valid bool
}

In an API setup using JSON and MySQL (i.e. My setup, but this is not exclusive), this structure allows me to easily deal with values that are "nullable" without having to deal with Pointers. I've always heard that it is best to avoid Pointers with the exception of pointers to structures (Slices, Slices of Structs, Map of Structs, etc). If you have a primitive type (int, bool, float, etc), try to avoid using a pointer to that primitive type.

This type has functions like MarshalJSON, UnmarshalJSON, Scan, and Value with logic inside those functions that leverage the Vaild property to determine what type of value to return. This works really really well with this setup.

Question

After some research, I've come to realize that Mongo would suit me better than a relational database, but due to the fluidity of a Mongo Document (Schemaless), I'm having a hard time understanding how to handle scenarios where a field maybe missing, or a property that i have in MySQL that would normally be null and I can easily unmarshal ontop this struct and use the helper functions logically, is handled. Also, when I setup a connection to Mongo and pull a couple of rows from MySQL and created Documents in Mongo from these rows, the BSON layer is marshalling the entire type for Alliance ID and sticking it in the DB.

Example:

    "alliance_id" : {
        "uint" : NumberLong(99007760),
        "valid" : true
    },

Where as in MySQL, the Value function implementing Valuer interface would be called and return 99007760 and that is the value in the DB.

Another scenario would be if valid was false. In MySQL this would mean a null value and when the Value function is called, it would return nil and the mysql driver would populate the field with NULL

So my question is how do I do this? Do I need to start from scratch and rebuild my models and redo some of the logic in my application that leverages the Valid property and use *Pointers or can I do what I am attempting to do using these helper types.

I do want to say that I have tried implementing the Marshaller, and Unmarshaller interfaces on the bson package and the alliane_id in the document is still set to the json encoded version of this type as I outlined above. I wanted to point this out to rule out any suggestions of implemeting those interfaces. If what I am attempting to achieve is counter intuitive to Mongo, please link some guides that can help me achieve what im attempting to do.

Thank you to all who can assist with this.

Upvotes: 1

Views: 430

Answers (1)

Burak Serdar
Burak Serdar

Reputation: 51657

The easiest way to deal with optional fields like this is to use a pointer:

type Character struct {
    ID               *uint64             `bson:"id,omitempty" json:"id"`
    Name              string             `bson:"name" json:"name"`
    ...
}

Above, the ID field will be written if it s non-nil. When unmarshaling, it will be set to a non-nil value if database record has a value for it. If you omit the omitempty flag, marshaling this struct will write null to the database.

For strings, you may use omitempty to omit the field completely if it is empty. If you want to store empty strings, omit omitempty.

Upvotes: 3

Related Questions