Reputation: 7
If using (@nestjs/schedule
)[https://docs.nestjs.com/techniques/task-scheduling] on a 2 cpu core server in pm2's cluster mode. It start 2 same task at once and cause data conflict in databse. How do you deal this?
Upvotes: 0
Views: 35
Reputation: 116
It would help if you introduced a coordination mechanism so that only one instance handles the scheduled job. A common solution is to use a distributed lock (e.g., Redis-based lock) or configure your code so that tasks run only on a designated “master” instance. This ensures that on a multi-process or clustered setup (like pm2 in cluster mode), only one process executes the task, preventing data conflicts.
When you run NestJS with @nestjs/schedule under pm2’s cluster mode, each CPU core spawns a separate Node.js process. Without coordination, every process will attempt to run the scheduled task simultaneously, causing duplicate work and data conflicts.
Only the first process that obtains the lock proceeds by using a distributed locking mechanism — such as a Redis-based lock — before starting the scheduled task. All other processes skip that run. Libraries like node-redlock can help implement this easily.
Alternatively, you might detect if the current process is the “primary” or “worker #0” and only run the scheduled task there. For example, you could set an environment variable on just one instance, or check cluster.isMaster before starting the scheduler. Only the primary instance executes the scheduled jobs, while others avoid it.
Another architectural approach is to run a dedicated instance solely responsible for scheduling tasks and all other instances act only as workers. This ensures tasks are never duplicated.
These patterns are common in clustered environments.
Upvotes: 0