CODEWITHSUNDEEP
node.jsmongodbcronlaterjsnode-celery
garima
garima

Reputation: 5244

Equivalent of Celery in Node JS

Please suggest an equivalent of Celery in Node JS to run asynchronous tasks. I have been able to search for the following:

  1. (Later)
  2. Kue (Kue),
  3. coffee-resque (coffee-resque)
  4. cron (cron)
  5. node-celery(node celery)

I have run both manual and automated threads in background and interact with MongoDB.

node-celery is using redis DB and not Mongo DB. Is there any way I can change that?When I installed node-celery redis was installed as dependency.

I am new to celery, Please guide.Thanks.

Upvotes: 31

Views: 35549

Answers (6)

Jojo Joseph Kunathucherry
Jojo Joseph Kunathucherry

Reputation: 384

  • Simple Solution

https://www.npmjs.com/package/node-cron

Code :

npm install --save node-cron

import cron from 'node-cron'

cron.schedule('* * * * *', () => {
  console.log('running a task every minute');
});

Upvotes: 0

securecurve
securecurve

Reputation: 5817

Edit-1/2018

My recommendation is not to use Kue now, as it seems to be a stalled project, use Celery instead. It is very well supported and maintained by the community and supports large number of use cases.

Old Answer

Go for Kue, it's a wholistic solution that resembles Celery in Python word; it has the concepts of: producers/consumers, delayed tasks, task retrial, task TTL, ability to round-robin tasks across multiple consumers listening to the same queue, etc.

Probably Celery is more advanced with more features with more brokers to support and you can use celery-node if you like, but, in my opinion, I think no need to go for a hybrid solution that requires installation of python and node when you can only use only language that's sufficient in 90% of the cases (unless necessary of course).

Upvotes: 7

jsbroks
jsbroks

Reputation: 550

It is also worth mentioning https://github.com/OptimalBits/bull. It is a fast, reliable, Redis-based queue written for stability and atomicity.

Bull 4 is currently in beta and has some nice features https://github.com/taskforcesh/bullmq

Upvotes: 7

Darvydas Šilkus
Darvydas Šilkus

Reputation: 329

Go for Kue, it's a wholistic solution that resembles Celery in Python word; it has the concepts of: producers/consumers, delayed tasks, task retrial, task TTL, ability to round-robin tasks across multiple consumers listening to the same queue, etc.

Kue, after so much time have passed, still has the same old core issues unsolved:

  • github.com/Automattic/kue/issues/514
  • github.com/Automattic/kue/issues/130
  • github.com/Automattic/kue/issues/53

If anyone reading this don't want to rewrite Kue, don't start with it. It's good for simple tasks. But if you want to deal with a lot of them, concurrent, or task chains (when one task creates another) - stop wasting your time.

I've wasted a month trying to debug Kue and still no success. The best choice was to change Kue for Pubs/sub Messaging queue on RabbitMQ and Rabbot (another RabbitMQ wrap up).

Personally, I haven't used Celery as much to put all in for it, but as I've been searching for Celery alternative and found how someone is advising for Kue just boiled my blood in veins.

If you want to send a delayed email (as in Kue example) you can go for whatever you'd like without worrying about errors. But if you want a reliable system task/message queue, don't even start with Kue. I'd personally go with 5. node-celery(node celery)

Upvotes: 6

Bryan Larsen
Bryan Larsen

Reputation: 10006

In our experience, Kue was unreliable, losing jobs. Granted, we were using an older version, it's probably been fixed since. That was also during the period when TJ abandoned the project and the new maintainers hadn't been chosen. We switched to beanstalkd and have been very happy. We're using https://github.com/ceejbot/fivebeans as the node interface to beanstalkd.

Upvotes: 2

Vanuan
Vanuan

Reputation: 33442

Celery is basically a RabbitMQ client. There are producers (tasks), consumers (workers) and AMQP message broker which delivers messages between tasks and workers.

Knowing that will enable you to write your own celery in node.js.

enter image description here

node-celery here is a library that enables your node process to work both as a celery client (Producer/Publisher) and a celery worker (Consumer).

See https://abhishek-tiwari.com/post/amqp-rabbitmq-and-celery-a-visual-guide-for-dummies

Upvotes: 20

Related Questions