I can't schedule a job inside scheduled cron job using node-schedule

Question

I have a cron job that runs every 5 seconds. Each time it runs I want to do some checks and based on that schedule a new job at a specific time if that job wasn't scheduled before. To do this I created a worker that contains the following code:

scheduleJob("worloads cron", "*/5 * * * * *", async function () {
  console.log("This job runs every 5 seconds");

  if (!connection) { // create DB connection if it doesn't exists
    Logger.info("Creating workloads DB connection.");
    connection = await createConnection({
      type: "postgres",
      url: process.env.DATABASE_URL,
      logging: true, // todo: disable in production
    });
    return Promise.resolve();
  } else if (connection && connection.isConnected) {

    // Fetch some data that needs to be scheduled
    const data: Workload[] = await connection.query(`
       SELECT workload."status"
       from "workload" workload;';
  `);

    console.log("jobs", scheduledJobs);

    for (const workload of data) {
      if (scheduledJobs[String(workload.id)]) {
        console.log(`Job with id ${workload.id} exists!`);
      } else {
        Logger.info(
          `Job with id ${workload.id} does not exist. Creating new scheduled job!`
        );
        //after this point my code doesn't execute and the job is not being scheduled
        const filter = await InteractionFilter.findOne({
          where: { id: workload.interactionFilter },
        });

        Logger.info("FILTER", filter); // this is no being logged
        const qb = await connection
          .getRepository(Interaction)
          .createQueryBuilder("interaction")
          .leftJoinAndSelect(
            `interaction.${filter.integrationSystem.toLowerCase()}_metadata`,
            "metadata"
          )
          .where("interaction.system = :system", {
            system: filter.integrationSystem,
          })
          .orderBy("interaction.createdAt", "ASC")
          .getMany();

        const filter_values = await connection
          .getRepository(InteractionFilter_Metadata)
          .createQueryBuilder("filtervalue")
          .leftJoinAndSelect("filtervalue.metadata", "metadata")
          .where("filtervalue.filterId = :filterId", {
            filterId: workload.interactionFilter,
          })
          .distinct()
          .getMany();

        Logger.info("FILTER VALUES", filter_values);

        // console.log(qb);

        scheduleJob(
          String(workload.id),
          dayjs(workload.executeTime).toDate(),
          function () {
            // todo: here create the interactions
            console.log("TEST");
          }
        );
        console.log("SCHEDULE TASK AFTER", scheduledJobs);
      }
    }

    return Promise.resolve();
  }
});


expose(() => true);

Here is my output, as you can see after we check that the job is not scheduled the function is not executing the code after and instead goes back to the start.

INFO [12-05-2023 16:22:40]: Creating workloads DB connection.
This job runs every 3 seconds
query: 
       SELECT workload."status"
       from "workload" workload;

jobs {
  'worloads crong': Job {

  }
}
INFO [12-05-2023 16:22:50]: Job with id 53 does not exist. Creating new scheduled job!

// HERE SINCE THE JOB DOESN'T EXIST IT SHOULD BE SCHEDULED INSTEAD I GET THE FOLLOWING OUTPUT:

This job runs every 3 seconds
query: 
       SELECT workload."status"
       from "workload" workload;

jobs {
  'worloads crong': Job {
    ...
  }
}
INFO [12-05-2023 16:23:00]: Job with id 53 does not exist. Creating new scheduled job!

Only way I was able to make it work is by commenting all the promises as such:

scheduleJob("worloads crong", "*/10 * * * * *", async function () {
  console.log("This job runs every 3 seconds");
  if (!connection) {
    Logger.info("Creating workloads DB connection.");
    connection = await createConnection({
      type: "postgres",
      url: process.env.DATABASE_URL,
      logging: true, 
    });
    return Promise.resolve();
  } else if (connection && connection.isConnected) {
            const data: Workload[] = await connection.query(`
       SELECT workload."status"
       from "workload" workload;';
  `);


    console.log("jobs", scheduledJobs);

    for (const workload of data) {
      if (scheduledJobs[String(workload.id)]) {
        console.log(`Job with id ${workload.id} exists!`);
      } else {
        Logger.info(
          `Job with id ${workload.id} does not exist. Creating new scheduled job!`
        );


        scheduleJob(
          String(workload.id),
          dayjs(workload.executeTime).toDate(),
          function () {
            // todo: here create the interactions
            console.log("TEST");
          }
        );
        console.log("SCHEDULE TASK AFTER", scheduledJobs);
      }
    }

    return Promise.resolve();
  }
});

And now I get my desired output:

    INFO [12-05-2023 16:27:40]: Creating workloads DB connection.
    This job runs every 3 seconds
    query: 
           SELECT workload."status"
           from "workload" workload;
    
    jobs {
      'worloads crong': Job {
      }
    }
    SCHEDULE TASK AFTER {
      '53': Job {
...
      },
      'worloads crong': Job {
...      
}
    }
    INFO [12-05-2023 16:27:50]: Job with id 53 does not exist. Creating new scheduled job!
    This job runs every 3 seconds
        query: 
           SELECT workload."status"
           from "workload" workload;
    jobs {
      '53': Job {
       ...
      },
      'worloads crong': Job {
       ...
      }
    }
    Job with id 53 exists!

Does anyone have an idea on how to fix this?

Answer 1

Hard to tell exactly what's wrong but I can suggest ways to move forward with your investigation.

The promise you indicated is not running might be error'd. I recommend you wrap your async operations in try-catches in order to be able to catch any errors that might be popping up. It's a good practice in general to handle errors gracefully.

Side note, be careful with scheduled jobs on a short timeout (like the 5s you're using), especially when you have asynchronous operations which might lag an otherwise fast operation.