CODEWITHSUNDEEP
neo4jpy2neo
kunal
kunal

Reputation: 35

Py2neo (Neo4j) insert / create Relationships in bulk

Trying to insert many new Relationship on existing nodes. Current code is taking too much time for millions of Relationship. Is there a way to optimise the same?

from py2neo import *

g = Graph()
nodes = NodeMatcher(g)

for persons in relation:
    person_a, person_b = persons

    a = nodes.match("person",name=person_a).first()
    b = nodes.match("person",name=person_b).first()

    ab = Relationship(a, 'KNOWS' b)
    ab['date'] = '01-01-1980'

    g.create(ab)

Now assume 2 things:

  1. Relations are in millions
  2. To process things faster, I have a pickle dump which consists of all the node details in py2neo.data.Node datatype so that we can skip the nodes.match(...) part.

Note: If there is any other way to create the complete graph faster in bulk mode (where I'm willing to create the entire graph from scratch if time taken by adding Relationships > time taken to create entire graph). Number of nodes are around 80K.

Upvotes: 4

Views: 1492

Answers (2)

hwong557
hwong557

Reputation: 1439

I found the bulk API is quick if you do not use the start_node_key and end_node_key parameters and instead directly specify the ID.

Upvotes: 0

Nigel Small
Nigel Small

Reputation: 4495

Py2neo has a bulk API that you might find useful: https://py2neo.org/2021.1/bulk/index.html

Upvotes: 1

Related Questions