CODEWITHSUNDEEP
elasticsearchamazon-ec2aws-lambdaamazon-vpcaws-networking
George Joseph
George Joseph

Reputation: 29

AWS Lambda function can't query elastic search running on ec2 instance

I have a lambda function which is trying to read data from elastic search running in an ec2 machine. Both of them are in the same VPC, subnet and have the same security group assigned to it. The Lambda can't seem to access the elastic search instance.

const AWS = require('aws-sdk');
const elasticsearch = require('elasticsearch');

exports.handler = async function (event, context, callback) {

  const client = new elasticsearch.Client({
    host: 'public_dns:9200',
    httpAuth: 'user:password'
  });

  let self = this;

  client.search({
    index: 'index_name',
    scroll: '30s',
    size: 10000,
    body: {
      query: {
        match_all: {}
      }
    }
  })
    .then(response => {
      self.responseString = response.hits.hits;
      console.log(response.hits.hits);
    })
    .catch(error => {
      console.error(error);
    });

  const responseData = {
    statusCode: 200,
    body: JSON.stringify({
      message: self.responseString
    })
  };

  callback(null, responseData);
};

The error i get from lambda is

2023-02-06T23:19:54.890Z    fcd62836-4fe3-4c6a-9871-ee70668ba07c    ERROR   StatusCodeError: Request Timeout after 30000ms
    at /var/task/node_modules/elasticsearch/src/lib/transport.js:397:9
    at Timeout.<anonymous> (/var/task/node_modules/elasticsearch/src/lib/transport.js:429:7)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7) {
  status: undefined,
  displayName: 'RequestTimeout',
  body: undefined

Any tips on how to debug this would be highly appreciated. Note: I do not wish to make the elastic search endpoint accessible to public.

Upvotes: 0

Views: 258

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 270294

Merely putting two resources "in the same Security Group" does not mean that they are able to communicate with each other. In fact, resources are not 'inside' a Security Group -- rather they are associated with a Security Group.

Security Group rules are applied to each resource individually. This means that if both resources are associated with the same security group, there needs to be a specific rule that allows incoming access from the security group to itself.

Instead of using one Security Group, a preferred configuration would be:

  • A security group on the AWS Lambda function (lambda-SG) with the default "Allow All" outbound rules, and
  • A security group on the Amazon EC2 instance running Elasticsearch (elastic-SG) that permits Inbound access on port 9200 from lambda-SG

That is, elastic-SG specifically references lambda-SG when permitting the inbound access. This means that traffic from the Lambda function will be allowed to communicate with the EC2 instance on that port.

Upvotes: 2

Related Questions