CODEWITHSUNDEEP
goneo4jneoism
Hokutosei
Hokutosei

Reputation: 2164

Insert thousand nodes to neo4j using golang

I am importing data to neo4j using neoism, and I have some issues importing big data, 1000 nodes, would take 8s. here is a part of the code that imports 100nodes. quite basic code, needs improvement, anyone can help me improve this?

var wg sync.WaitGroup
for _, itemProps := range items {
    wg.Add(1)
    go func(i interface{}) {
        s := time.Now()
        cypher := neoism.CypherQuery{
            Statement: fmt.Sprintf(`
                CREATE (%v)
                SET i = {Props}
                RETURN i
            `, ItemLabel),
            Parameters: neoism.Props{"Props": i},
        }
        if err := database.ExecuteCypherQuery(cypher); err != nil {
            utils.Error(fmt.Sprintf("error ImportItemsNeo4j! %v", err))
            wg.Done()
            return
        }
        utils.Info(fmt.Sprintf("import Item success! took: %v", time.Since(s)))
        wg.Done()
    }(itemProps)
}
wg.Wait()

Upvotes: 0

Views: 2067

Answers (1)

Michael Hunger
Michael Hunger

Reputation: 41676

Afaik neoism still uses old APIs, you should use cq instead: https://github.com/go-cq/cq

also you should batch your creates,

i.e. either send multiple statements per request, e.g 100 statements per request

or even better send a list of parameters to a single cypher query:

e.g. {data} is a [{id:1},{id:2},...]

UNWIND {data} as props
CREATE (n:Label) SET n = props

Upvotes: 4

Related Questions