CODEWITHSUNDEEP
aws-lambdaamazon-sqs
RSevier
RSevier

Reputation: 45

How to Trigger an AWS Lambda Function from External SQS Queue Activity

I'm trying to configure a lambda function to consume from an SQS queue that I've been given read and delete permissions to, that I do not own/have configuration of. Is there a way to use lambda's SQS trigger functionality for a queue that doesn't exist inside my AWS account?

If not, what are some alternatives that don't include checking the queue on a scheduled event.

Upvotes: 3

Views: 689

Answers (1)

MyStackRunnethOver
MyStackRunnethOver

Reputation: 5275

If the owner of the SQS queue gives you the necessary permissions (see the setup docs for what those permissions are), you can do this. But, you shouldn't.

Subscribing to someone else's SQS queue is an anti-pattern. This is because a queue represents a backlog of work, and the implicit functionality is that everything that goes in eventually comes out. All the queue does is separate input flow from output flow (data can flow in both faster and slower than they flow out).

This idea of flow, however, means that when something comes out, it's no longer in the queue. (Caveat here: there are work-arounds to this, but they're usually not preferred). A consumer, however, always has the goal of processing everything in the queue. This may be done by multiple threads under the control of one consumer, but the end result is still that everything is processed. If there are multiple consumers, then they by necessity compete with one another, and none of them get to process everything in the queue.

How do we ensure there aren't multiple consumers? Simple: the consumer owns the queue. No other consumer is granted read permissions. It might well be the case that someone other than the consumer controls the filling of the queue (receiving write permissions) - and AWS has the perfect solution for this:

SNS Topics: An SNS topic is a source of data. It is, in effect, a publisher. When someone else wants you to have access to their data, they allow you to become a subscriber to their topic. When a new message is published to the SNS topic, everyone who is subscribed to the topic gets a copy. What happens to that copy is decided by the subscriber: it may be acted upon directly, stored for later action, or acted on indirectly, e.g. by being placed in a queue. This is the Pub-Sub model. It separates the details of one entity (the publisher) creating messages and sending them out to many others, from each recipient's (subscriber's) individual decision about how to consume those messages.

TL;DR: get whoever is currently owning the queue to publish to an SNS topic instead, then set up a queue (or whatever you prefer) subscribed to that topic.

Upvotes: 1

Related Questions