CODEWITHSUNDEEP
neo4j
mukherjeejayant
mukherjeejayant

Reputation: 153

Getting incremental changes from Neo4j DB

Is there any way in Neo4j 1.9, to get all the nodes/relationships that were modified(created/updated/deleted) within certain span of time - like we do in SOLR delta import?

One crude way I can think of is maintain a timestamp property for each node/relationship and index them to fetch those nodes/relationship.

START a=node:custom_index("timestamp:[{start_time} TO {end_time}]")
RETURN a;

But then the question would be if I modify the node via CYPHER, index will not be updated.

Upvotes: 0

Views: 415

Answers (1)

Michal Bachman
Michal Bachman

Reputation: 2661

There's no built-in functionality like that in Neo4j, unfortunately.

To address issues one by one. Maintaining timestamp is not possible, because you have nowhere to put it in the case of deleted nodes/relationships. You can't put a timestamp on a property either. So you would know a node has been changed, but wouldn't know how.

One possible solution is to log the changes somewhere as they happen, using TransactionEventHandlers. Then, you can a) choose exactly what to record, and b) don't worry about Cypher, it will be logged no matter what method you used to update the database.

I've put together a small demo. It just logs every change to std out. It uses some GraphAware classes (disclaimer: I'm the author) for simplicity, but could be written without them, if you feel so inclined.

Here's the important part of the code, in case the link gets eventually broken or something:

@Test
public void demonstrateLoggingEveryChange() {
    GraphDatabaseService database = new TestGraphDatabaseFactory().newImpermanentDatabase();

    database.registerTransactionEventHandler(new ChangeLogger());

    //perform mutations here
}

private class ChangeLogger extends TransactionEventHandler.Adapter<Void> {

    @Override
    public void afterCommit(TransactionData data, Void state) {
        ImprovedTransactionData improvedData = new LazyTransactionData(data);

        for (Node createdNode : improvedData.getAllCreatedNodes()) {
            System.out.println("Created node " + createdNode.getId()
                    + " with properties: " + new SerializablePropertiesImpl(createdNode).toString());
        }

        for (Node deletedNode : improvedData.getAllDeletedNodes()) {
            System.out.println("Deleted node " + deletedNode.getId()
                    + " with properties: " + new SerializablePropertiesImpl(deletedNode).toString());
        }

        for (Change<Node> changedNode : improvedData.getAllChangedNodes()) {
            System.out.println("Changed node " + changedNode.getCurrent().getId()
                    + " from properties: " + new SerializablePropertiesImpl(changedNode.getPrevious()).toString()
                    + " to properties: " + new SerializablePropertiesImpl(changedNode.getCurrent()).toString());
        }

        for (Relationship createdRelationship : improvedData.getAllCreatedRelationships()) {
            System.out.println("Created relationship " + createdRelationship.getId()
                    + " between nodes " + createdRelationship.getStartNode().getId()
                    + " and " + createdRelationship.getEndNode().getId()
                    + " with properties: " + new SerializablePropertiesImpl(createdRelationship).toString());
        }

        for (Relationship deletedRelationship : improvedData.getAllDeletedRelationships()) {
            System.out.println("Deleted relationship " + deletedRelationship.getId()
                    + " between nodes " + deletedRelationship.getStartNode().getId()
                    + " and " + deletedRelationship.getEndNode().getId()
                    + " with properties: " + new SerializablePropertiesImpl(deletedRelationship).toString());
        }

        for (Change<Relationship> changedRelationship : improvedData.getAllChangedRelationships()) {
            System.out.println("Changed relationship " + changedRelationship.getCurrent().getId()
                    + " between nodes " + changedRelationship.getCurrent().getStartNode().getId()
                    + " and " + changedRelationship.getCurrent().getEndNode().getId()
                    + " from properties: " + new SerializablePropertiesImpl(changedRelationship.getPrevious()).toString()
                    + " to properties: " + new SerializablePropertiesImpl(changedRelationship.getCurrent()).toString());
        }
    }
}

Upvotes: 2

Related Questions