CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-sqs
Jwan622
Jwan622

Reputation: 11639

AWS Lambda times out when trying to connect to sqs

My lambda times out when trying to connect to SQS. The Lambda has a role that allows it to connect to SQS, but it is located inside a VPC and subnet. What might be the solution to this problem?

I see this error in my logs:

[INFO]  2019-10-01T14:29:58.303Z    e8ad5b4e-119a-48c1-b320-1d855c4efb22    Getting SQS queue url from <some_sqs_queue>...
    14:30:16
[CRITICAL]  2019-10-01T14:30:16.743Z    e8ad5b4e-119a-48c1-b320-1d855c4efb22    ## Transmission Error Connect timeout on endpoint URL: "https://us-west-2.queue.amazonaws.com/"

and this is the relevant code which works locally but not up on the lambda:

 sqs = boto3.client(
        'sqs', # region_name="us-west-2",
        aws_access_key_id=credentials.access_key,
        aws_secret_access_key=credentials.secret_key,
        aws_session_token=credentials.token,
        config=Config(connect_timeout=6, read_timeout=10, retries={'max_attempts': 2})
    )

and

try:
    logger.info(f"Getting SQS queue url from {sqs_queue}...")
    queue_url = sqs.get_queue_url(QueueName=sqs_queue)['QueueUrl']
    # iterate over entries in batches of 10
    for batch in [entries[index:index + sqs_batch_limit] for index in range(0, len(entries), sqs_batch_limit)]:
        logger.info(f"Sending batch of {len(batch)} records to sqs...")
        sqs.send_message_batch(
            QueueUrl=queue_url,
            Entries=batch
        )
        logger.info("Sent batch of records to sqs successful")

It doesn't even get the sqs_queue_url but it works locally. What are the possible reasons for this log Transmission Error Connect error?

Upvotes: 3

Views: 3746

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269330

You can that the AWS Lambda function "is located inside a VPC and subnet".

However, Amazon SQS exists on the Internet, not in the VPC.

Therefore, you will either need a way for the Lambda function to reach the Internet, or a way to make Amazon SQS available within the VPC.

Option 1: Allow the AWS Lambda function to reach the Internet

When a Lambda function is not configured to use a VPC, it can communicate directly with the Internet. However, when it is configured to use a VPC, it does not have direct access to the Internet.

Instead, the recommended configuration is:

  • Attach the Lambda function to the private subnets in the VPC
  • Launch a NAT Gateway in a public subnet and update the private Route Table to send Internet-bound traffic via the NAT Gateway

Option 2: Make Amazon SQS accessible within the VPC

You can create an Amazon VPC Endpoint for Amazon SQS in the VPC:

If you use Amazon VPC to host your AWS resources, you can establish a connection between your VPC and Amazon SQS. You can use this connection to send messages to your Amazon SQS queues without crossing the public internet.

Upvotes: 9

Related Questions