CODEWITHSUNDEEP
node.jsamazon-web-servicesamazon-sqslong-polling
Suhail Gupta
Suhail Gupta

Reputation: 23206

How deal with the multiple messages from SQS?

The following function will receive multiple messages from the sqs. Each message has to be processed and the database updated accordingly.

I can deal with a single message, by calling the pull function from the worker module. But how to deal with the multiple messages? I cannot keep calling the pull method of the worker module in a loop, since it will block the thread. What is the best way possible here?

function checkMessage(){
    var params = {
                QueueUrl : Constant.QUEUE_URL,
                VisibilityTimeout: 0,
                WaitTimeSeconds: 20,
                MaxNumberOfMessages: 10
            }
    sqs.receiveMessage(params,(err,data) => {
        if(data){
            var workerId = uuidV4();
            // Now worker will pull the message for processing
            // The worker response is returned in the callback function
            Worker.pull(data,workerId,(err,respData) => {
                if(respData){
                    // If the message was successfully processed
                    // set the final job status to complete and 
                    // progress to 100%
                }else{
                    // If the processing failed set the final
                    // job status to error
                }
            });
        }
    });
}

Pull method from Worker module:

function pull(messageObject,workerId,cb){
    if(messageObject){
        var messageProcessed = true;
        /* 
         * Process the message as required. Before starting the processing
         * set the job status to processing.
         */

        /**
         * After the message has been processed, call the callback function
         * inside monitor module.
         */
        var callbackObject = {jobid : jobId, serverid : workerId};
        if(messageProcessed){
            return cb(null,callbackObject);
        }else {
            return cb(new Error('Failed to process messgae'),callbackObject);
        }
    }
}

Upvotes: 1

Views: 724

Answers (1)

Jason Livesay
Jason Livesay

Reputation: 6377

None of your code shown is synchronous or CPU intensive. So I would test if you actually have that problem. If the code not shown is synchronous or CPU instensive, you may have a problem whether there is a loop or not. So you could use a separate thread with webworker-threads or another process. If you need to just process in order search npms.io for 'queue'.

Upvotes: 1

Related Questions