CODEWITHSUNDEEP
node.jslambdaredis
Ssander
Ssander

Reputation: 926

Reuse Redis connections for NodeJS Lambda function

I have written the following Lambda function:

exports.handler = (event, context, callback) => {
    const redis = require('redis');
    const redis_client = redis.createClient({
        host: 'hostname',
        port: 6379
    });

    redis_client.set("foo", "bar");

    redis_client.get("foo", function(err, reply) {
        redis_client.quit();
        callback(null, reply);
    });
};

This works fine. However, I would like to reuse the Redis connection between Lambda invocations to improve performance. In theory this would be possible by moving the createClient() to outside of the handler. However, because of the "redis_client.quit()" line, that connection is killed. If I do not quit the client, the Lambda function will time out however.

What is the proper way to reuse a Redis in NodeJS when using AWS Lambda?

Upvotes: 13

Views: 7533

Answers (4)

Simon Harper
Simon Harper

Reputation: 29

Rather than creating the connection for each invocation. A better pattern would be to declare a variable in the global scope then check if empty inside the handler.

This way you are able to establish the connection once per execution environment.

const redis = require('redis');

let redisClient;

exports.handler = (event, context, callback) => {
  context.callbackWaitsForEmptyEventLoop = false;

  if (!redisClient) {
    redisClient = redis.createClient({
      host: 'hostname',
      port: 6379
    });
  }

  redisClient.set("foo", "bar");

  ...
}

https://docs.aws.amazon.com/lambda/latest/operatorguide/global-scope.html https://github.com/sequelize/sequelize/issues/4938#issuecomment-245211042

Upvotes: 0

Justin Kruse
Justin Kruse

Reputation: 1130

To avoid the timeout, set callbackWaitsForEmptyEventLoop to false in the AWS Context of your Lambda. This way you do not have to close the redis connection and your lambda will not wait for it (or other connections, i.e. db connections) to close before returning:

https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html

Upvotes: 2

Tom
Tom

Reputation: 61

To reuse the Redis client connection define it as a global outside the handler.

const redis = require('redis');
const redis_client = redis.createClient({
    host: 'hostname',
    port: 6379
});

exports.handler = (event, context, callback) => {
    redis_client.set("foo", "bar");

    redis_client.get("foo", function(err, reply) {
        redis_client.unref();
        callback(null, reply);
    });
};

Upvotes: 6

Risto Novik
Risto Novik

Reputation: 8295

As you already mentioned your approach is correct way but you have to keep in mind that your Redis instance has connection limits, example AWS Elasticache maxclients is set to 65000. AWS currently allows executing 1k Lambdas in parallel so be careful with the external connections.

Currently, there is no silver bullet for Lambda external DB connections. One possible solution would be to create internal web API service which handles the communication between the DB.

Upvotes: 4

Related Questions