CODEWITHSUNDEEP
neo4jcypher
sranga
sranga

Reputation: 123

Long time in load_csv

One of my cypher query to create relations takes very long time. Running for hours and did not complete.

I have loaded the nodes through load_csv which are roughly around 22M.

id is the unique index of all nodes.

Pin label is around 16M.

CREATE INDEX ON :Pin(id);
CREATE INDEX ON :Instance(id);

//CREATE CONSTRAINT ON (inst:Instance) ASSERT inst.Id IS UNIQUE;

USING PERIODIC COMMIT 10000
LOAD CSV WITH HEADERS FROM 'file:///home/sranga/work/Neo4j/work/signals.csv' AS line
MATCH (inst1:Pin { id: toInteger(line.Start_port)})
MATCH (inst2:Pin { id: toInteger(line.End_port)})
CREATE (inst1)-[:NET { id: toInteger(line.Signal_id),  Name: line.Name, Signed: toInteger(line.Signed), lbit: toInteger(line.Lbit), rbit: toInteger(line.Rbit)}]->(inst2);

Any idea as what is wrong?

Upvotes: 1

Views: 219

Answers (1)

Jerome_B
Jerome_B

Reputation: 1097

Use EXPLAIN and PROFILE to get a picture of how the query is run. Then add it to your question. (ok I saw you copy/pasted, and it does not talk much)

Is your constraint applied or not ? Are the indexes created ?

Try to reduce 10 000 to 1000. When this number is too high, you get outofmemoryerrors. That should commit every 1000 lines read. Follow the progress by connecting a cypher-shell to your instance and run a query to count the NET relationships. Something like MATCH ()-[net:NET]->() return count(net); in order to verify that it is increasing (else you stop it, breathe, and send another message )

check the number of lines in your csv with 

 cat file.csv |wc -l

to be able to calculate the end of your import

Upvotes: 1

Related Questions