CODEWITHSUNDEEP
neo4j
Shihab Rahman
Shihab Rahman

Reputation: 53

Faster way to insert data in neo4j?

I am trying to insert unique nodes and relationship in neo4j.

What I am using :-

Data Size and Type :

TSV File [Multiple]. Each contains more than 8 Million Lines [each line represent a node or a relationship].There are more than 10 files for nodes.[= 2 Million Nodes] and another 2 million relations.

I am using UniqueNodeFactory for inserting nodes. And inserting sequentially, couldn't find any way to insert into batches preserving unique nodes.

The problem is it is taking huge time to insert data. For example it took almost a day for inserting 0.3 million unique nodes. Is there any way to fasten the insertion?

Upvotes: 1

Views: 1650

Answers (1)

Michael Hunger
Michael Hunger

Reputation: 41676

Don't do that.

Java-REST-Binding was never made for that.

Use either 

USING PERIODIC COMMIT 1000
LOAD CSV WITH HEADERS FROM "http://some.url" as line
CREATE (u:User {name:line.name})

You can also use merge (with constraints), create relationships etc.

See my blog post for an example: http://jexp.de/blog/2014/06/using-load-csv-to-import-git-history-into-neo4j/ Or the Neo4j Manual: http://docs.neo4j.org/chunked/milestone/cypherdoc-importing-csv-files-with-cypher.html

Upvotes: 5

Related Questions