Monitoring real-time changes to results of a neo4j query?

Question

I have an incoming event stream of player interactions from an MMO. I want to construct a graph of the player's moment-to-moment interactions, continuously run queries on the activities of the past ~30-240 seconds, and update a graphical view, all in real-time.

---

Some more details about my particular case:

• I have ~100-500 incoming events every second. These look like:

  (PlayerA)->[:TAKES_ACTION]->(Event)->[:RECIPIENT]->(PlayerB)

where every event is time-stamped. Timestamps are accurate to the second. I plan on attaching every event to a node representing a timestamp, so I can restrict queries to the events attached to a set of X most recent timestamps.

• I expect at any given time-frame for there to be ~1000-2000 active players.

• My queries will be to group players together based on mutual interactions, to figure out which groups are currently engaged in combat with which other groups.

---

My main questions are:

• Does Neo4j have any sort of "incremental update" functionality to efficiently update query results without re-running the entire query for every set of changes?

• Does Neo4j have any sort of ability to "push" any changes to the results of a query to a client? Or would a client have to continuously poll the database?

• Are there any optimisations or tricks to making a continuously repeated query as efficient as possible?

Answer 1

Answers

1) No. You can only execute query and get results.

2) No. Currently you can only make client -> server requests.

3) Yes.

Details

Let's get to the bottom of this one. Neo4j by default can offer you:

• REST API
• Transactional Cypher ednpoint
• Traversal endpoint
• Custom plugins
• Custom unmanaged extensions

In your case you should implement unmanaged extension. This is best option to get desired functionality - develop it by yourself.

More information on extensions:

• How to create unmanaged Neo4j extension?
• Unmanaged extension template
• Graphaware framework for extension development

In extension you can do everything you want:

• Use Core API to make efficient queries for latest data
• Create WebSocket endpoint for full-duplex communication channel between client and server
• Implement any additional logic to format/represent your data correctly

Queries and performance

Cypher queries are compiled and cached on first execution. After that - cached query version is used. And query execution by itself is quite fast.

Recommendations:

• Always use query parameters where it is possible. This allow Neo4j to efficiently reuse queries
• Be smart on writing queries. Try to lower cardinality where possible.
• Think about data model. Probably you can model your data in such way, when query always fetches only latest data. In you case probably relationship :LAST_EVENT, :PREVIOUS_EVENT and etc. can help.