CODEWITHSUNDEEP
neo4j
Tim Ferrell
Tim Ferrell

Reputation: 1366

All queries are slow with neo4j

I have written a variety of queries using cypher that take no less than 200ms per query. They're very straightforward, so I'm having trouble identifying where the bottleneck is.

Simple Match with Parameters, 2200ms:

SimpleMatchWithParameters

Simple Distinct Match with Parameters, 200ms:

SimpleDistinctMatchWithParameters

Pathing, 2500ms:

enter image description here

At first I thought the issue was a lack of resources, because I was running neo4j and my application on the same box. While the performance monitor indicated that CPU and memory were largely free'd up and available, I moved the neo4j server to another local box and observed similar latency. Both servers are workstations with fairly new Xeon processors, 12GB memory and SSDs for the data storage. All of the above leads me to believe that the latency isn't due to my hardware. OS is Windows 7.

The graph has less than 200 nodes and less than 200 relationships.

I've attached some queries that I send to neo4j along with the configuration for the server, database, and JVM. No plugins or extensions are loaded.

Pastebin Links:

Upvotes: 5

Views: 4933

Answers (3)

cybersam
cybersam

Reputation: 67044

[Expanding a bit on a comment I made earlier.]

@TFerrell: Your comments state that "all nodes have labels", and that you tried applying indexes. However, it is not clear if you actually specified the labels in your slow Cypher queries. I noticed from your original question statement that neither of your slower queries actually specified a node label (which presumably should have been "Project").

If your Cypher query does not specify the label for a node, then the DB engine has to test every node, and it also cannot apply an index.

So, please try specifying the correct node label(s) in your slow queries.

Upvotes: 4

Michael Hunger
Michael Hunger

Reputation: 41706

It seems there is something wrong with the automatic memory mapping calculation when you are on Windows (memory mapping on heap).

I just looked at your messages.log and added up some numbers, so it seems the mmio alone is enough to fill your java heap space (old-gen) leaving no room for the database, caches etc.

Please try to amend that by fixing the mmio config in your conf/neo4j.properties to more sensible values (than the auto-calculation). For your small store just uncommenting the values starting with #neostore. (i.e. remove the #) should work fine.

Otherwise something like this (fitting for a 3GB heap) for a larger graph (2M nodes, 10M rels, 20M props,10M long strings):

 neostore.nodestore.db.mapped_memory=25M
 neostore.relationshipstore.db.mapped_memory=250M
 neostore.propertystore.db.mapped_memory=250M
 neostore.propertystore.db.strings.mapped_memory=250M
 neostore.propertystore.db.arrays.mapped_memory=0M

Here are the added numbers:

auto mmio: 134217728 + 134217728 + 536870912 + 536870912 + 1073741824 = 2.3GB
stores sizes: 1073920 + 1073664 + 3221698 + 3221460 + 1073786 = 9MB

JVM max: 3.11 RAM : 13.98 SWAP: 27.97 GB
max heaps: Eden: 1.16, oldgen: 2.33

taken from:

neostore.propertystore.db.strings] brickCount=8 brickSize=134144b mappedMem=134217728b (storeSize=1073920b)
neostore.propertystore.db.arrays] brickCount=8 brickSize=134144b mappedMem=134217728b (storeSize=1073664b)
neostore.propertystore.db] brickCount=6 brickSize=536854b mappedMem=536870912b (storeSize=3221698b)
neostore.relationshipstore.db] brickCount=6 brickSize=536844b mappedMem=536870912b (storeSize=3221460b)
neostore.nodestore.db] brickCount=1 brickSize=1073730b mappedMem=1073741824b (storeSize=1073786b)

Upvotes: 2

Michael Hunger
Michael Hunger

Reputation: 41706

Is that the first run or a subsequent run of these queries?

You probably don't have a label on your nodes and no index or unique constraint.

So Neo4j has to scan the whole store for your node pulling everything into memory, loading the properties and checking.

try this:

run until count returns 0:

match (n) where not n:Entity set n:Entity return count(*);

add the constraint

create constraint on (e:Entity) assert e.Id is unique;

run your query again:

match (n:Element {Id:{Id}}) return n

etc.

Upvotes: 4

Related Questions