CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaaws-cloudformationamazon-cloudwatch
JohnJ
JohnJ

Reputation: 4823

Specify log group for an AWS lambda?

Is there a way to specify the CloudWatch log group that an AWS lambda logs to? It seems to be generated directly from the lambda name; however, it would be especially convenient to, for example, aggregate multiple lambdas to a single log group. We are especially interested in specifying the log group when the lambda is created by a CloudFormation template.

Upvotes: 91

Views: 85442

Answers (7)

jarmod
jarmod

Reputation: 78583

As of November 2023, you can now specify a custom CloudWatch Logs log group name when creating a Lambda function.

In CloudFormation for your Lambda function you can indicate the log group name, as follows:

Type: AWS::Lambda::Function
Properties:
  FunctionName: <my function name>
  LoggingConfig:
    ApplicationLogLevel: TRACE
    LogFormat: <optional custom format>
    LogGroup: <custom log group name>
    SystemLogLevel: WARN

Note that you can also indicate the application log level (DEBUG | ERROR | FATAL | INFO | TRACE | WARN) and system log level (DEBUG | INFO | WARN) which you previously could not do, as well as the log format.

Upvotes: 7

Noah Gary
Noah Gary

Reputation: 960

Okay so I had to do this myself and @pat-myron commented a link to the way you do it below the answer.

He posted a feature request where @benbridts outlines the template here.

Here is the gist of what worked in my cloudformation template:

HelloWorldLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: "/aws/lambda/HelloWorld"
      RetentionInDays: 30

HelloWorldFunction:
  Type: AWS::Lambda::Function
  DependsOn: HelloWorldLogGroup
  Properties:
    FunctionName: HelloWorld
    Role:
      Fn::GetAtt:
        - LambdaExecRole
        - Arn

LambdaExecRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: "2012-10-17"
      Statement:
        - Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
          Action: sts:AssumeRole
    Policies:
      - PolicyName: LambdaLogging
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - logs:CreateLogStream
                - logs:PutLogEvents
              Resource: !GetAtt HelloWorldLogGroup.Arn

Upvotes: 5

sziraqui
sziraqui

Reputation: 6092

I attached LogGroup to Serverless function in SAM template as follows:

    MyFuncLogGroup:
        Type: AWS::Logs::LogGroup
        Properties:
            LogGroupName: '/aws/lambda/stackName-env-myFunc-v1'
            RetentionInDays: 30

    MyFunc:
        Type: AWS::Serverless::Function
        Properties:
            FunctionName: 'stackName-env-myFunc-v1'
            ...

Some users mentioned the requirement of DependsOn property but it is not required in my experience when using SAM. The only requirement is that the LogGroupName must be /aws/lambda/<FunctionName>. SAM will create the log group before the lambda function as long as your are not referencing the function's logical ID in the LogGroup

Also, if you are adding LogGroup to exisiting function, you can do so by simply updating your template.yaml with the LogGroup resource like above and add FunctionName property to the function resource. Obviously, the FunctionName should be different from your existing lambda function name that appears in your lambda's ARN.

WARNING: FunctionName property requires replacement. Your existing lambda will be deleted and new one will be created.

Upvotes: 5

Tanvir
Tanvir

Reputation: 1522

I find that @lingrlongr's answer is partially correct.

First, to answer the original question, you cannot specify a custom log group name for the lambda function to write to.

The lambda log group name always follows this pattern: 

/aws/lambda/<function-name>

The lambda function will first check if a log group exists with this name.

  • If it exists, then it will use that.
  • If it does not exist, it will create a log group with that pattern.

Hence, if you want to add settings, such as RetentionInDays and SubscriptionFilter, make sure your CloudFormation or SAM template creates the LogGroup before the lambda function. If your lambda function is created first, an error will be thrown when creating the LogGroup saying that the LogGroup already exists. So, the lambda function should have DependsOn: LogGroup instead of the other way round.

Also, make sure you are not using Ref or GetAtt to reference the lambda function inside the LogGroup because that creates an implicit dependency on the lambda function causing the lambda function being created before the LogGroup.

Upvotes: 25

cardella
cardella

Reputation: 129

Creating the log group as mentioned as one of the answers works. To keep the retention policy after the stack is deleted, just add a DeletionPolicy.

"MyLambdaFunctionLogGroup": {
  "Type": "AWS::Logs::LogGroup",
  "DependsOn": "MyLambdaFunction",
  "DeletionPolicy": "Retain",
  "Properties": {
    "LogGroupName": {"Fn::Join": ["", ["/aws/lambda/", {"Ref": "MyLambdaFunction"}]]},
    "RetentionInDays": 14
  }
}

Upvotes: 9

lingrlongr
lingrlongr

Reputation: 459

Actually, maybe you can, to an extent at least. I too was in search of the answer and gave this a try. Here's a snippet of two resources; the lambda function and the log group:

"MyLambdaFunction": {
    "Type": "AWS::Lambda::Function",
    "DependsOn": "ReadWriteRole",
    "Properties": {
        //snip
    }
},

"MyLambdaFunctionLogGroup": {
    "Type": "AWS::Logs::LogGroup",
    "DependsOn": "MyLambdaFunction",
    "Properties": {
        "LogGroupName": {"Fn::Join": ["", ["/aws/lambda/", {"Ref": "MyLambdaFunction"}]]},
        "RetentionInDays": 14
    }
},

I found that the log group was created with a retention of 14 days as indicated. When the lambda function runs, it does create log streams in this group. When I deleted the stack, however, it seems that the log groups is not deleted, and the retention is now set to never expire. Perhaps that's good enough so the streams don't get too out of hand...

Upvotes: 44

Leon
Leon

Reputation: 32474

I don't think that is possible.

Even if it were possible, each AWS Lambda instance would still write to its own log-stream. And though different invocations of the same lambda can write to the same log-stream (when the lambda instance is reused), this will definitely not be the case for different lambdas (since they must use different lambda instances).

As a result, you must have a tool that aggregates multiple log-stream. If so, what's the problem with making it a little more generic, so that it can aggregate log-streams from different log groups?

Upvotes: 32

Related Questions