CODEWITHSUNDEEP
jsongoelasticsearch
josephkibe
josephkibe

Reputation: 1343

How do I pass through values from an Elasticsearch document in Go?

I'm experimenting with rewriting parts of our system in Go. They're currently written in Python. Some of the data that I want to serve lives in Elasticsearch.

Across our users, we have a few standard fields, but also allow people to create a number of custom fields specific to their environment. E.g., we have a product object that has some common fields like name and price, but we let someone create a field like discount_price or requires_freight to agree with their use case.

In Python, this is easy to accommodate. The JSON is read in, our chosen JSON parser does some reasonable type inference, and we can then return the data after it's processed.

In Go, any data we want to deal with from the Elasticsearch JSON response has to be mapped to a type. Or at least that's my understanding. For example:

import (
    "encoding/json"
)

...

type Product struct {
    Name              string   `json:"name"`
    Price             string   `json:"price"`
}

Here's a simplified example of what the data might look like. I've prefixed the names of the nonstandard fields I'd want to pass through with custom:

{
    "id": "ABC123",
    "name": "Great Product",
    "price": 10.99,
    "custom_alternate_names": ["Great Product"],
    "custom_sellers": [{
        "id": "ABC123",
        "name": "Great Product LLC",
        "emails": ["[email protected]"]
    }]
}

It would be fine to special case for places where the route actually needs to process or manipulate a custom field. But in most cases, passing through the JSON data unchanged is fine, so the fact we aren't imposing any type mappings for safety isn't adding anything.

Is there a way to set up the struct with an interface (?) that could act as a passthrough for any unmapped fields? Or a way to take the unparsed JSON data and recombine it with the mapped object before the data are returned?

Upvotes: 1

Views: 521

Answers (1)

Pablo Flores
Pablo Flores

Reputation: 1400

You can do something like this

package main

import (
    "encoding/json"
    "log"
)

type Product struct {
    //embed product to be able to pull the properties without
    //requiring a nested object in the json
    KnownFields
    OtherStuff json.RawMessage
}

//Custom unmarshaller
func (p *Product) UnmarshalJSON(b []byte) error {
    var k KnownFields
    //unmarshal the known fields
   err := json.Unmarshal(b, &k)
    if err != nil {
        return err
    }
    p.KnownFields = k
    //You can use json.RawMessage or map[string]interface
    p.OtherStuff = json.RawMessage(b)
    return nil
}

type KnownFields struct {
    Name  string      `json:"name"`
    Price json.Number `json:"price"`
}

const JSON = `
{
    "id": "ABC123",
    "name": "Great Product",
    "price": 10.99,
    "custom_alternate_names": ["Great Product"],
    "custom_sellers": [{
        "id": "ABC123",
        "name": "Great Product LLC",
        "emails": ["[email protected]"]
    }]
}`

func main() {
    var p Product
    err := json.Unmarshal([]byte(JSON), &p)
    if err != nil {
        log.Panic(err)
    }
    log.Printf("%v", p)
}

If you are going to mutate and marshall the Product you will have to implement a custom marshaller and also you would need to use map[string]interface{} as the OtherStuff to avoid having duplicate entries for the known fields

Upvotes: 1

Related Questions