CODEWITHSUNDEEP
sqljsonpostgresqlgo
sravis
sravis

Reputation: 3680

How to handle nullable Postgres JSONB data and parse it as JSON

DB Records

---------------------------------------------------------
| id | test_json                                        |
---------------------------------------------------------
| 1  | NULL                                             |
---------------------------------------------------------
| 2  | { "firstName": "Hello", "lastName": "World" }    |
---------------------------------------------------------

I have JSONB column in postgres which can be NULL. I want to read this records DB in golang and send it to the client.

I'm getting below error on SQL scan:

sql: Scan error on column index 2, name "test_json": unsupported Scan, storing driver.Value type []uint8 into type *models.TestJSONNullable
exit status 1

I'm using echo web server.

package models

import (
    "fmt"
    "github.com/lib/pq"
    "encoding/json"
)

type TestJson struct {
    First_name *string `json:"firstName"`
    Last_name *string `json:"lastName"`
}

type TestJSONNullable struct {
  Valid bool
}

func (i *TestJSONNullable) UnmarshalJSON(data []byte) error {
  if string(data) == "null" {
    i.Valid = false
    return nil
  }

  // The key isn't set to null
  var temp *TestJson
  if err := json.Unmarshal(data, &temp); err != nil {
    return err
  }
  i.Valid = true
  return nil
}

type Test01 struct {
    Id string `json:"id"`
    Test_json *TestJSONNullable `json:"testJson"`
}


func (db *DB) TestRecords () ([]*Test01, error)  {
    rows, err := db.Query("SELECT id, test_json FROM table_1 where success = true")

    if err != nil {
        log.Fatal(err)
        return nil, err
    }

    defer rows.Close()

    recs := []*Test01{}

    for rows.Next() {
        r := new(Test01)

        err := rows.Scan(&r.Id, &r.Test_json)

        if err != nil {
            log.Fatal(err)
            return nil, err
        }

        recs = append(recs, r)
    }

    if err = rows.Err(); err != nil {
        log.Fatal(err)
        return nil, err
    }

    return recs, nil
}

Upvotes: 3

Views: 4179

Answers (3)

Soroush
Soroush

Reputation: 1105

Here's another solution: You can implement a nullable type for raw JSON data, similar to sql.NullString, and use it as a scan destination. In this scenario, you will first check whether the value is null, and then unmarshal it only if it's not null. For instance, the NullRawMessage type from github.com/soroushj/sqlt is a nullable json.RawMessage which can be used for this purpose.

Here's an example:

package main

import (
    "database/sql"
    "log"

    _ "github.com/lib/pq"
    "github.com/soroushj/sqlt"
)

func main() {
    db, err := sql.Open("postgres", "dbname=dbname user=user password=password sslmode=disable")
    if err != nil {
        log.Fatal(err)
    }
    row := db.QueryRow(`SELECT test_json FROM my_table WHERE id = $1`, 1)
    testJSON := sqlt.NullRawMessage{}
    err = row.Scan(&testJSON)
    if err != nil {
        log.Fatal(err)
    }
    if testJSON.Valid {
        // test_json is not null
        // Unmarshal testJSON.RawMessage
    } else {
        // test_json is null
    }
}

Upvotes: 2

sravis
sravis

Reputation: 3680

After doing some research i found the solution.

type TestJSONMap map[string]interface{}

func (t TestJSONMap) Value() (driver.Value, error) {
    j, err := json.Marshal(t)
    return j, err
}

func (p *TestJSONMap) Scan(val interface{}) error {
    value, ok := val.([]byte)
    if !ok {
        return errors.New("Type assertion .([]byte) failed.")
    }

    var i interface{}
    err := json.Unmarshal(value, &i)
    if err != nil {
        return err
    }

    *p, ok = i.(map[string]interface{})
    if !ok {
        return errors.New("Type assertion .(map[string]interface{}) failed.")
    }

    return nil
}

type Test01 struct {
    Id string `json:"id"`
    Test_json *TestJSONMap `json:"testJson"`
}

Got help from https://coussej.github.io/2016/02/16/Handling-JSONB-in-Go-Structs/

Upvotes: 0

arshpreet
arshpreet

Reputation: 729

Instead of using struct to store values, You can use map.

type TestJson struct {
    First_name *string `json:"firstName"`
    Last_name *string `json:"lastName"`
}

You can use interface as

var TestJson interface{}
err := json.Unmarshal(b, &TestJson)

On the other side you can also use Dynamic creation of structs as well. Something like

m := map[string]interface{}{
    "key": "value",
}

And rather than having TestJSONNullable it would be better to have Switch case while saving data.

switch v := TestJson.(type) {
case int:

case float64:

case string:

default:
    // i isn't one of the types above
}

Look into this for more details--> https://godoc.org/encoding/json#Unmarshal

Upvotes: -1

Related Questions