CODEWITHSUNDEEP
javascheduled-tasksscheduler
EnE_
EnE_

Reputation: 541

Scheduling tasks to prevent overlap of processing and bandwidth usage in Java

I am creating a daemon that will run certain scheduled tasks for logging but I am worried about bottle necking certain points.

Effectively I have some logging tasks that I want to execute every 15 minutes and some I only want to execute every 30 minutes and so on up to tasks that only need running once a month. Basically I have a list of checks to make at each time interval. These are put into a queue and processed by a thread pool.

At the moment I see the tasks as running something like this

15
15, 30
15, 30, 60
15, 30, 60, 120
15, 30, 60, 120, 240...

This means that if the daemon starts at 00:00 hours then by 04:00 hours there will be five processes running simultaneously and this is not the end of it. At present this has led the next task scheduled for 15 minutes to run slowly and have access to a restricted amount of bandwidth.

It is not neccessary for the tasks to run on the hour however. So if the 15 minute task runs on the hour the 30 minute may start at 5 minutes past the hour so as to minimise overlap. It would even be possible to split the two 30 minutes tasks (e.g. 00:00 and 00:30) across the four 15 minute process to reduce being hit by an 'all at once' type problem but this really gets my head swimming.

Are there any well known methodologies for managing this type of issue?

Upvotes: 0

Views: 1658

Answers (2)

instanceOfObject
instanceOfObject

Reputation: 2994

Well, for long run (as I see from your question), you will have to look for something like quartz.

Apart from this, I have couple of more concerns and suggestions here:

  1. Use [ScheduledExecutorService][2] for managing these threads. Even with ScheduledExecutorService, looks like you want to run them at separate interval irrespective of their execution time. SchedulewithFixedRate rather than ScheduleWithFixedDelay.

  2. Even if you implement anything like this, your logic will fail if your threads start to run on multiple hosts. 2 hosts running hourly threads will effectively run every 30 minutes.

  3. I would prefer to have a centralized management in terms of Database to keep track of last run and all. This along with ExecutorService would be scalable and accurate.

Suppose you have 1000 schedules, create 1000 rows in the DB.

Columns will somewhat be like this

id, P_Key
ScheduleName, AnyIdentifier for the daemon to run or task to do.
lastRunTime, lastTime it was run.
granularity, 15 mins, 30 mins etc.

You can keep CreationTime and ModificationTime as best practices.

Upvotes: 0

Anders R. Bystrup
Anders R. Bystrup

Reputation: 16070

You should definitely have a look at Quartz, especially the cron style triggers.

Cheers,

Upvotes: 0

Related Questions