CODEWITHSUNDEEP
node.jsmongoosecron
program_bumble_bee
program_bumble_bee

Reputation: 305

Rescheduling cron jobs in nodejs on server restart

I have node js application where a user can schedule 4-5 different cron jobs. A user selects a start_date, end_date, start_time, and end_time, and accordingly, the cron functions are scheduled and some campaigns are generated to connect with leads (or other users). I am using npm package node-schedule to achieve the desired cron-job execution but the issue arises if my web application crashes in between.

As per the official documentation of node-schedule,

Note that Node Schedule is designed for in-process scheduling, i.e. scheduled jobs will only fire as long as your script is running, and the schedule will disappear when execution completes.

The solution I can come up to resolve this is to call the cron functions using server.js file which restarts every time the server script is executed. But I am unable to resolve some of the following issues:

  1. A scheduler is an individual function, not campaign-specific, so how will the application know which cron-function to start if it has not been completed?
  2. A scheduler is a single function, that will be called every time as soon as the server is restarted. If 6 instances of a server are created, it will be called 6 times.

Suppose, my scheduler is due to execute between 28th May 11:00 AM to 31st May 12:00 AM. The scheduler starts its job and some of the campaigns are executed in between. If my server goes down on 30th May, how can I pick up from the remaining date-time and campaigns and complete it from there?

So I am concerned about how to schedule or handle the cron job functions for the campaigns? If any of the campaign cron jobs is crashed in between if the server is stopped, how should I be able to re-schedule or execute that cron function from the time or date that has been left of it?

I am not much experienced with cron jobs, hoping for some handy solutions.

Upvotes: 1

Views: 6971

Answers (1)

JRichardsz
JRichardsz

Reputation: 16524

Here some advice, that you could review or implement depending of your hardware capabilities:

Decoupling

Decouple your current application in two separated artifacts: web and campaign. This allows each service to be deployed, rebuilt, re-deployed and managed independently. And also to have autonomous or cross-functional Teams.

source: https://skelia.com/articles/5-major-benefits-microservice-architecture/

Fault tolerance

If you need an app that doesn't affect a failure , crash or any event who needs a restart, you must use something outside to your app.

Commonly this outside is a database. You could use a json file if a database is not an option for you.

Campaign

We have two entities here:

campaign_configurations

| identifier | desc | cron_expression | |-------------------|----------------|-----------------| | month_discounts | discounts ... | 0 0 1 * * | | reminder_shop_car | a reminder ... | */10 * * * * | | | | | Also you can add a column to control the period validity of the cron with a start and end date.

campaign_executions

| execution_date | execution_status | execution_log | |---------------------|------------------|---------------| | 10-10-2020 01:00:00 | finished | ... | | 10-10-2020 14:30:00 | error | ... | | 10-10-2020 14:35:00 | executing | ... |

App startup

When your nodejs campaign starts/restarts due to failure or scheduled maintance, this must be the flow:

  • read campaign_configurations table and get all of cron expressions in a valid period.
  • use an algorithm to detect if today your cron expressions must be executed. Example:
    • a cron only in December (Christmas) should not run today: June 03
    • a cron for every 5 minutes, should execute in the next 5 minutes
  • Every execution at the start must create an entry in campaign_executions table with a status: pending
  • Every execution could use node-schedule to execute it logic.
  • New executions should not execute if there is a previous execution in pending or error. According to your internal business logic.

App is healthy

When app is healthy, its must query to the campaign_configurations table in order to execute new or modified cron expressions (previous explained flow)

This could be at every 30 minutes, more and less, according to your internal business logic.

You could use node-schedule for this task.

Upvotes: 3

Related Questions