CODEWITHSUNDEEP
node.jsaws-lambdaamazon-rdsamazon-rds-proxy
kenshin9
kenshin9

Reputation: 2375

Lambda Database Connections - Best Practices

I've been looking over Lambdas for this new code base I'm working in. We're currently connecting directly to our RDS Aurora instances in our Lambdas, and I was looking into the best practices and started noticing areas that I think could be improved. I started going down a bit of a rabbit hole to make sure I understood most of this, but wanted to validate some thoughts I had and get insight from those that have actually done this.

One step at a time, but the thing that stood out to me immediately is that we initialize a connection to the database inside the handler function and it is always closed at the end. So this happens on every invocation. I've read many times that it's recommended to reuse these connections by initializing the connection outside the handler function.

If the connection is initialized once, I had wondered what would happen if the connection was dropped for whatever reason during the lifetime of the Lambda, and if it was invoked again. And I eventually got on the subject of RDS Proxy. In the end, I'm going to suggest that we look into using RDS Proxy.

Now back to the Lambdas. I had a few questions:

  1. Am I correct that we should be initializing the database connections outside the handler method? That way other invocations would reuse the connection.
  2. How do you authenticate the connection to the proxy? Do you use secrets in Secrets Manager or do you use IAM authentication? We currently use secrets right now, but the secret values are resolved on every invocation of the Lambda too, so I was thinking that was something we could reduce as well. I had read that IAM authentication would be a good benefit since we don't have to manage secrets in the Lambda, and the execution role would just need the right policy. But in the Lambda, we would still need to request an authToken. So in terms of request overhead, it doesn't sound like we would escape that.

Upvotes: 6

Views: 6236

Answers (1)

Dewaun Ayers
Dewaun Ayers

Reputation: 743

Am I correct that we should be initializing the database connections outside the handler method?

This is absolutely correct. This is stated in the lambda best practices guidelines in the official AWS docs:

Take advantage of execution environment reuse to improve the performance of your function. Initialize SDK clients and database connections outside of the function handler...

You can fully connect outside the handler or use a let connection declaration outside of your handler, and configure your connection inside the handler and get similar results. For the latter, you would just need a condition inside the handler to check if the connection is undefined and if so, configure it!

How do you authenticate the connection to the proxy? Do you use secrets in Secrets Manager or do you use IAM authentication?

I personally opt for using Secrets Manager every time. You can get a performance boost here by caching the secret values using the Params and Secrets Lambda Layer detailed here

This extension layer works for both the Parameter Store and Secrets Manager. It creates a local server that queries Secrets Manager and stores the results of your request in cache for a period of time. Subsequent Lambda invocations will hit the cache instead of Secrets Manager while the cache is still valid. You can set environment variables in the lambda to tweak things like cache size and ttl.

simple layer extension diagram

If you are using the AWS CDK then you can configure the layer using the ParamsAndSecretsOptions Lambda construct config:

const paramsAndSecrets = lambda.ParamsAndSecretsLayerVersion.fromVersion(lambda.ParamsAndSecretsVersions.V1_0_103, {
  cacheSize: 500,
  logLevel: lambda.ParamsAndSecretsLogLevel.DEBUG,
});

const lambdaFunction = new lambda.Function(this, 'MyFunction', {
  runtime: lambda.Runtime.NODEJS_18_X,
  handler: 'index.handler',
  architecture: lambda.Architecture.ARM_64,
  code: lambda.Code.fromAsset(path.join(__dirname, 'lambda-handler')),
  paramsAndSecrets,
});

Hope this helps!

Upvotes: 5

Related Questions