CODEWITHSUNDEEP
jsonpointersgostruct
odzhu
odzhu

Reputation: 173

Persisting nested structs in golang

I want to add persistence and initialize from json. I'm trying to save/load nested structs and getting "fatal error: stack overflow".

As I understand, the reason is that both parent and child structs have pointers to each other and json library is going into loops. I need pointer to Parent since need access it from the child.

I understand this is common issue, what is the best way to address it?

type Mcloud struct {
    Projects map[string]*Project `json:"Projects"`
    Workdir  string
}

type Project struct {
    Name     string
    Networks map[string]Network
    Parent   *Mcloud
    TFC      TFConf
}

func newMcloud() *Mcloud {
    mc := &Mcloud{
        Projects: make(map[string]*Project),
        Workdir:  defaultWorkDir,
    }
    mc.load(statefile)

    return mc
}


func (mc *Mcloud) addProject(n string) {
    mc.Projects[n] = &Project{
        Name:     n,
        Networks: make(map[string]Network),
        Parent:   mc,
    }
    mc.Projects[n].addTFConf()
}

//save saves state to statefile
func (mc *Mcloud) save(f string) (err error) {
    if jsonState, err := json.Marshal(mc); err != nil {
        fmt.Println("Was not able to marshal")
        log.Fatal(err)
    } else {
        if err := ioutil.WriteFile(f, jsonState, 0666); err != nil {
            fmt.Println("Was not able to write state to", f, "!")
            log.Fatal(err)
        }
        fmt.Println("Save function saves: \n", mc, "to file ", f)
    }

    return err
}

func (mc *Mcloud) load(f string) (err error) {
    var bytestate []byte

    if bytestate, err = ioutil.ReadFile(f); err == nil {
        err = json.Unmarshal(bytestate, &mc)
    }

    return err
}

Getting

runtime: goroutine stack exceeds 1000000000-byte limit fatal error: stack overflow

runtime stack: runtime.throw(0x149cdfe, 0xe) /usr/local/Cellar/go/1.11.1/libexec/src/runtime/panic.go:608 +0x72 runtime.newstack() /usr/local/Cellar/go/1.11.1/libexec/src/runtime/stack.go:1008 +0x729 runtime.morestack() /usr/local/Cellar/go/1.11.1/libexec/src/runtime/asm_amd64.s:429 +0x8f

goroutine 1 [running]: runtime.heapBitsSetType(0xc042a7df20, 0x60, 0x60, 0x1486e60) /usr/local/Cellar/go/1.11.1/libexec/src/runtime/mbitmap.go:911 +0xa30 fp=0xc02243c388 sp=0xc02243c380 pc=0x1016cd0 runtime.mallocgc(0x60, 0x1486e60, 0x1, 0x0) /usr/local/Cellar/go/1.11.1/libexec/src/runtime/malloc.go:933 +0x540 fp=0xc02243c428 sp=0xc02243c388 pc=0x100d4f0 runtime.newobject(0x1486e60, 0x0) /usr/local/Cellar/go/1.11.1/libexec/src/runtime/malloc.go:1032 +0x38 fp=0xc02243c458 sp=0xc02243c428 pc=0x100db28 reflect.mapiterinit(0x14206a0, 0xc00009d830, 0x0)

Upvotes: 0

Views: 1252

Answers (1)

mkopriva
mkopriva

Reputation: 38213

First you need to tell encoding/json to skip the parent field, you can do that with the json:"-" tag.

Then during unmarshaling, after you've loaded all of a parent's children you loop over the children and set their parent, you can do this as part of the unamrshaling process by implementing the json.Unmarshaler interface.

type Mcloud struct {
    Projects map[string]*Project `json:"Projects"`
    Workdir  string
}

type Project struct {
    Name     string
    Networks map[string]Network
    Parent   *Mcloud `json:"-"` // ignore on un/marshal
    TFC      TFConf
}

func (m *Mcloud) UnmarshalJSON(data []byte) error {
    type tmp Mcloud
    if err := json.Unmarshal(data, (*tmp)(m)); err != nil {
        return err
    }

    // set Parent of all projects
    for _, p := range m.Projects {
        p.Parent = m
    }
    return nil
}

Upvotes: 1

Related Questions