CODEWITHSUNDEEP
neo4jcypherneo4j-apoc
Sreenath S
Sreenath S

Reputation: 35

How to quickly warmup Neo4J page cache for a set of nodes and relationships

I'm setting up a REST API backed by a Neo4J instance of complex data about 400 GB of store size and need to warmup page cache for a set of nodes and relationships.

There are specific calls for a set of input data which takes a lot of time to query and get a response. So I want a way to warmup the cache only for the nodes and relationships which are accessed for these calls. I tried using APOC.warmup.run(true, true) (took around 15 mins which is acceptable for me), but it does load the whole store to memory which I can't do since I have a constraint on memory.I tried writing a simple Cypher which traverses through these path, but it is taking lot of time to execute and when I check the memory growth of Neo4J instance which is very slow as compared to APOC warmup.

I am also thinking if there is a way to extend/customize APOC warmup to load only specific parts of the store, but want to see if there are people out there who already tried something similar before.

I expect a quick way of warming up specific part of the store rather than the whole store.

Upvotes: 0

Views: 554

Answers (1)

Michael Hunger
Michael Hunger

Reputation: 41706

You should also have enough page-cache configured for that (400G)

If you use apoc.warmup.run(true,true,true) also the schema indexes will be warmed up.

I guess it only took 15 minutes for you because your disk read performance might be lower? And how many CPUs do you have?

You cannot really load a specific part of the store.

But recent Neo4j versions track page-cache usage and restore it after restart. So you can just run your queries and after a restart the same pages will be in the page cache.

You also see with PROFILE if you have page-faults in your query.

Upvotes: 1

Related Questions