CODEWITHSUNDEEP
javaneo4jcypherdatabasenosql
Muhammad Adnan
Muhammad Adnan

Reputation: 490

Neo4j graph database java.lang.OutOfMemoryError: Java heap space. Neo4j graph database

I have successfully migrated dblp dataset in to neo4j database and i use neo4jShell for running the cypher quires. The database has millions of nodes and relations between publications and authors. Now when i try to run a query on neo4j database it takes 10 to 12 hours for processing and then ended up with this error

Error occurred in server thread; nested exception is : java.lang.OutOfMemoryError: Java heap space

i am using neo4j community edition version 2.2.3, jdk 1.7 machine with 8 gb of memory and core i7 processor.

Query :

neo4j-sh (?)$ MATCH (p:`publication`)-[:`publishedby`]->(a:`author`)
RETURN p.year, p.type, a.id, count(*) order by a.id desc LIMIT 25;

Experts please advice me any way out from this exception.

Upvotes: 3

Views: 4243

Answers (2)

Michael Hunger
Michael Hunger

Reputation: 41706

As your dataset is a public dataset it would be very helpful if you could share your database.

In general you are computing many million or billion paths, which you are aggregating after the fact, that just takes a while. Combined with probably too little memory and a slow disk it takes a long time to load the data from disk.

This is a global graph query, you can see that if you run it prefixed with PROFILE.

Make sure your id property is numberic !

I would change the query like this:

// this is the expensive operation, to order millions of authors by id
// still, do it and take the top 25
MATCH (a:author) WITH a order by a.id LIMIT 25
// find publications for the top 25 authors
MATCH (a)<-[:publishedby]-(p)
// return aggregation
RETURN a.id, p.year, p.type, count(*)
LIMIT 25;

To start neo4j-shell with sensible memory settings:

  • stop the server
  • edit conf/neo4j-wrapper.conf, set min and maxmemory to 4000
  • edit conf/neo4j.properties set dbms.pagecache.memory=3G
  • start the server, run bin/neo4j-shell

if you run neo4j-shell in standalone mode, stop the server and use this:

export JAVA_OPTS="-Xmx4000M -Xms4000M -Xmn1000M" 
bin/neo4j-shell -path data/graph.db -config conf/neo4j.properties

Upvotes: 1

Ricardo Vila
Ricardo Vila

Reputation: 1632

Probably you should set more max memory to your java process. Java process only uses up to the max memory amount configured, by default it is ussually only 256 MB. Use -Xmx parameter to achieve that. Read this How to set the maximum memory usage for JVM? to have more detailed explanation.

Be aware that you must use a 64 bit jdk, and 64 bit OS to set Xmx to more than 4 GB.

Upvotes: 1

Related Questions