How do I unmarshal JSON into a Go struct provided by a different piece of code?

Question

I am writing a Go library that will decode JSON into a struct. The JSON has a fairly simple common schema, but I want consumers of this library to be able to decode additional fields into their own structs that embed the common struct, avoiding the need to use maps. Ideally, I'd like to decode the JSON only once.

Currently it looks something like this. (Error handling removed for brevity.)

The JSON:

{ "CommonField": "foo",
  "Url": "http://example.com",
  "Name": "Wolf" }

The library code:

// The base JSON request.
type BaseRequest struct {
    CommonField string
}

type AllocateFn func() interface{}
type HandlerFn func(interface{})

type Service struct {
    allocator AllocateFn
    handler HandlerFn
}   

func (Service *s) someHandler(data []byte) {
    v := s.allocator()
    json.Unmarshal(data, &v)
    s.handler(v)
}

The app code:

// The extended JSON request
type MyRequest struct {
    BaseRequest
    Url string
    Name string
}

func allocator() interface{} {
    return &MyRequest{}
}

func handler(v interface{}) {
    fmt.Printf("%+v\n", v);
}

func main() {
    s := &Service{allocator, handler}
    // Run s, eventually s.someHandler() is called
}

The thing I don't like about this setup is the allocator function. All implementations are simply going to return a new BaseRequest "sub-type". In a more dynamic language I would pass the type of MyRequest in instead, and instantiate inside the library. Do I have a similar option in Go?

Answer 1

Another way I came up with is to use reflection.

Tweaking my original example, the library code becomes:

// The base JSON request.
type BaseRequest struct {
    CommonField string
}

type HandlerFn func(interface{})

type Service struct {
    typ reflect.Type
    handler HandlerFn
}   

func (Service *s) someHandler(data []byte) {
    v := reflect.New(s.typ).Interface()
    json.Unmarshal(data, &v)
    s.handler(v)
}

and the app code becomes:

// The extended JSON request
type MyRequest struct {
    BaseRequest
    Url string
    Name string
}

func handler(v interface{}) {
    fmt.Printf("%+v\n", v);
}

func main() {
    s := &Service{reflect.TypeOf(MyRequest{}), handler}
    // Run s, eventually s.someHandler() is called
}

I haven't decided if I like this any better. Maybe the way to go is just to simply unmarshal the data twice.

Answer 2

There are several ways to handle this. One idea which is both simple and convenient is defining a richer Request type that you provide to the handler, instead of handing off the raw type. This way you can implement the default behavior in a friendly way, and support the edge cases. This would also avoid the need to embed the default type on custom types, and allow you to expand functionality without breaking clients.

For inspiration:

type Request struct {
    CommonField string

    rawJSON []byte
}

func (r *Request) Unmarshal(value interface{}) error {
    return json.Unmarshal(r.rawJSON, value)
}

func handler(req *Request) {
    // Use common data.
    fmt.Println(req.CommonField)

    // If necessary, poke into the underlying message.
    var myValue MyType
    err := req.Unmarshal(&myValue)
    // ...
}

func main() {
    service := NewService(handler)
    // ...
}

Answer 3

I think json.RawMessage is used to delay decoding subsets of JSON. In your case you can maybe do something like this:

package main

import (
↦       "encoding/json"
↦       "fmt"
)

type BaseRequest struct {
↦       CommonField string
↦       AppData json.RawMessage
}

type AppData struct {
↦       AppField string
}

var someJson string = `
{
↦       "CommonField": "foo",
↦       "AppData": {
↦       ↦       "AppField": "bar"
↦       }
}
`

func main() {
↦       var baseRequest BaseRequest
↦       if err := json.Unmarshal([]byte(someJson), &baseRequest); err != nil {
↦       ↦       panic(err)
↦       }

↦       fmt.Println("Parsed BaseRequest", baseRequest)

↦       var appData AppData
↦       if err := json.Unmarshal(baseRequest.AppData, &appData); err != nil {
↦       ↦       panic(err)
↦       }

↦       fmt.Println("Parsed AppData", appData)
}