CODEWITHSUNDEEP
amazon-web-servicesamazon-ec2aws-sdkamazon-sqs
Jonathan Plackett
Jonathan Plackett

Reputation: 2426

Amazon AWS Worker does not delete messages from SQS queue

As I understand it my AWS elastic beanstalk Workers subscribing to an SQS queue should automatically delete messages if I send a 200 OK header.

A web application in a worker environment should only listen on the local host. When the web application in the worker environment returns a 200 OK response to acknowledge that it has received and successfully processed the request, the daemon sends a DeleteMessage call to the SQS queue so that the message will be deleted from the queue. (SQS automatically deletes messages that have been in a queue for longer than the configured RetentionPeriod.) If the application returns any response other than 200 OK, then Elastic Beanstalk waits to put the message back in the queue after the configured VisibilityTimeout period. If there is no response, then Elastic Beanstalk waits to put the message back in the queue after the InactivityTimeout period so that the message is available for another attempt at processing.

But mine are not doing that. They are processing OK and saving the video they're encoding to the right S3 bucket. But they do not delete the queue item like they should.

I have tried sending the header various ways including...

http_response_code(200);

header("HTTP/1.1 200 OK");

header("HTTP/1.1 200 OK", true, 200);

I have no output on the page but have tried calling ob_start(); at the start and ob_end_flush(); after sending the header, and even tried just doing the header right at the start before any processing. Nothing works and the messages remain in flight and get redelivered after their visibility timeout ends.

I can't think of anything else to delete these messages - the worker (I think) only receives the message body so I can't even take matters into my own hands and delete the message by its handler using the API.

Help!

Thanks.

Upvotes: 3

Views: 1732

Answers (1)

Jonathan Plackett
Jonathan Plackett

Reputation: 2426

OK I have realised this has nothing to do with sending or not sending the response code.

The code was being sent but the daemon to to actually clear the queue item wasn't running because I was using a disk image for a normal Elastic beanstalk application (I had been using this before and assumed they would be the same - they are not)

To fix, I created a new elastic beanstalk worker app, took the standard Custom AMI ID from configuration>instances and searched for it in the community instances and spun up a new ec2 instance replica.

I made my customisations to this new Worker style server and made an image of it, got that custom ID and replaced the worker ID, spun down the ec2 server and deleted that new worker app I got the ID from.

Once I rebooted the original app now the messages get deleted.

I hope this helps someone at some point!

Upvotes: 4

Related Questions