CODEWITHSUNDEEP
jsongo
Ice3man
Ice3man

Reputation: 125

Parsing multiple JSON objects in Go

Objects like the below can be parsed quite easily using the encoding/json package.

[ 
    {"something":"foo"},
    {"something-else":"bar"}
]

The trouble I am facing is when there are multiple dicts returned from the server like this :

{"something":"foo"}
{"something-else":"bar"}

This can't be parsed using the code below.

correct_format := strings.Replace(string(resp_body), "}{", "},{", -1)
json_output := "[" + correct_format + "]"

I am trying to parse Common Crawl data (see example).

How can I do this?

Upvotes: 8

Views: 11125

Answers (4)

Peter
Peter

Reputation: 31691

Use a json.Decoder to decode them:

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "log"
    "strings"
)

var input = `
{"foo": "bar"}
{"foo": "baz"}
`

type Doc struct {
    Foo string
}

func main() {
    dec := json.NewDecoder(strings.NewReader(input))
    for {
        var doc Doc

        err := dec.Decode(&doc)
        if err == io.EOF {
            // all done
            break
        }
        if err != nil {
            log.Fatal(err)
        }

        fmt.Printf("%+v\n", doc)
    }
}

Playground: https://play.golang.org/p/ANx8MoMC0yq

Upvotes: 17

Manoj
Manoj

Reputation: 296

You can read the ndjson from the file row by row and parse it then apply the logical operations on it. In the below sample instead of reading from the file, I have used an Array of JSON string.

import (
    "encoding/json"
    "fmt"
)

type NestedObject struct {
    D string
    E string
}

type OuterObject struct {
    A string
    B string
    C []NestedObject
}

func main() {

    myJsonString := []string{`{"A":"1","B":"2","C":[{"D":"100","E":"10"}]}`, `{"A":"11","B":"21","C":[{"D":"1001","E":"101"}]}`}
    for index, each := range myJsonString {
        fmt.Printf("Index value [%d] is [%v]\n", index, each)
        var obj OuterObject
        json.Unmarshal([]byte(each), &obj)
        fmt.Printf("a: %v, b: %v, c: %v", obj.A, obj.B, obj.C)
        fmt.Println()
    }
}

Output:

Index value [0] is [{"A":"1","B":"2","C":[{"D":"100","E":"10"}]}]
a: 1, b: 2, c: [{100 10}]
Index value [1] is [{"A":"11","B":"21","C":[{"D":"1001","E":"101"}]}]
a: 11, b: 21, c: [{1001 101}]

Try it on golang play

Upvotes: 0

dm03514
dm03514

Reputation: 55962

Another option would be to parse each incoming line, line by line, and then add each one to a collection in code (ie a slice) Go provides a line scanner for this. 

yourCollection := []yourObject{}
scanner := bufio.NewScanner(YOUR_SOURCE)
for scanner.Scan() {
    obj, err := PARSE_JSON_INTO_yourObject(scanner.Text())
    if err != nil {
       // something
    }
    yourCollection = append(yourCollection, obj)
}
if err := scanner.Err(); err != nil {
    fmt.Fprintln(os.Stderr, "reading standard input:", err)
}

Upvotes: 0

Nebril
Nebril

Reputation: 3273

Seems like each line is its own json object.

You may get away with the following code which will structure this output into correct json:

package main

import (
    "fmt"
    "strings"
)

func main() {
    base := `{"trolo":"lolo"}
{"trolo2":"lolo2"}`

    delimited := strings.Replace(base, "\n", ",", -1)

    final := "[" + delimited + "]"
    fmt.Println(final)
}

You should be able to use encoding/json library on final now.

Upvotes: 2

Related Questions