Apache Ignite - Schedule In a Cluster

Question

I'm thinking on this since a week, but I came up with nothing that convinces me.

I have a small cluster running Jetty with Jersey on all nodes (they are all identical). Requests are balanced on nodes, and all nodes are "stateless".

User data is partitioned on Apache Ignite embedded in the JVMs with Jetty and therefore accessible to any node.

Till now all the actions performed on this data where fired by REST requests and managed by the node the request stumbled upon. But now requirements has changed. An example is as follow:

After a User perform a specific action (offer a virtual good), few other random users get notified about the opportunity to redeem this good. Upon action of any of them (within a defined time after publishing) they will own it. Here the problem: If no user redeem the good within the given time, I want to pick few new users and notify them.

This will require scheduling, I can see only two solutions:

• Schedule an event on the node that notified first few users. (Not reliable if node fail or get reloaded)

• Pool from all nodes a distributed collection where data required to trigger expiration events are stored. Then run those expired. (Looks like the less efficient solution when this collection grow)

  • A hibrid of both; where events are stored in a distributed collection and scheduled may be good. If node do not fail will run and delete the collection entry. A slow iteration of the collection will check if there are events that expired and has not been deleted. (It looks a solution that may be hard to debug and predict)

As well I have read the DelayedQueue implementation and do not look an efficient system to manage a great amount of scheduled tasks.

I will confess that I got stuck, and I need an advice on how events can be scheduled on a cluster basis being shure they will be called as wel as they will resist a node failure.

Answer 1

I think you can create a cluster singleton (https://apacheignite.readme.io/docs/cluster-singletons) that will take care of scheduling. This will automatically handle failover in case of crashes. If there is any state (e.g. information about the users that were notified, timestamp the job was scheduled last time, etc.), it can be saved in a cache.