amazon-web-servicesaws-lambdaamazon-cloudwatch

Reputation: 27783

Can't get AWS Lambda function to log (text output) to CloudWatch

I'm trying to set up a Lambda function that will process a file when it's uploaded to an S3 bucket. I need a way to see the output of console.log when I upload a file, but I can't figure out how to link my Lambda function to CloudWatch.

I figured about by looking at the context object that my log group is /aws/lambda/wavToMp3 and the log stream is 2016/05/23/[$LATEST]hex_code_redacted. So I created that group and stream in CloudWatch, yet nothing is being logged to it.

Upvotes: 120

Answers (18)

Cristan

Reputation: 14135

The lambda needs rights to write to CloudWatch, which the AWSLambdaBasicExecutionRole will take care off.

If that still doesn't work, try running the lambda manually with the Test button. You might get the error "Lambda was unable to decrypt the environment variables because KMS access was denied."

When that happens, change the IAM of the lambda to something else, save, and then change it back (github issue).

Upvotes: 1

Shibashis

Reputation: 8421

For the Lambda function to be able to create a log group and publish logs to that group, the execution role needs to have the following permissions:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Action": "logs:CreateLogGroup",
        "Resource": "arn:aws:logs:region:accountID:*"
    },

    {
        "Effect": "Allow",
        "Action": [
            "logs:CreateLogStream",
            "logs:PutLogEvents"
        ],
        "Resource": [
            "arn:aws:logs:region:accountID:log-group:/aws/lambda/functionname:*"
        ]
    }
]
}

Reference: https://docs.aws.amazon.com/lambda/latest/operatorguide/access-logs.html

Upvotes: 176

Suraj Sangani

Reputation: 1

For me, it turned out to be that the cloudwatch arn that the IAM role should point to should be arn:aws:logs:<region>:***:log-group:/aws/lambda/<lambda-function-name>:* instead of arn:aws:logs:<region>:***:log-group:/aws/lambda/<lambda-function-name>

Upvotes: 0

Guy

Reputation: 1332

In my case, the permission boundary set on the role was the issue.

Although the role policy allowed logs:PutLogEvents, the permission boundary set on the role disallowed this action on the log group.

Upvotes: 0

jellycsc

Reputation: 12359

July 2020 Update !!

Logs may not be in us-east-1, try looking for lambda edge logs in different regions !!

Upvotes: 11

santhosh

Reputation: 509

Though this got answered already, Just wanted to add my experience which might be useful for others.

Even the permissions are set appropriately so that lambda can log in cloudwatch, sometimes it is taking 3-4 hrs to reflect the log groups.

In my case when using lambda for DynamoDB events, where lambda was given all needed permissions for both cloudwatch and DynamoDB. It took 4 hrs to get the logs reflected. May be some sync issues from AWS end. After 4 hrs without any action from my end, I was able to see the Cloudwatch logs.

Also make sure that you are seacrching the logs in the same region where your function is created.

Upvotes: 0

C.M.

Reputation: 1561

You might want add the Last Ingestion Time column to the log output. It's taking a few minutes for all my events to be written.

Upvotes: 0

hoonoh

Reputation: 2271

After you update your policy, it seems that you have to update your function's settings to refresh all job instances to read new policies.

So if you just click 'test' button from Lambda console after you update your role policy in IAM, the cached Lambda instances will still have old role permissions, so you will still see no logs being written to Cloudwatch logs.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

Upvotes: 216

mbarlocker

Reputation: 1386

There's a writeup called How to Monitor AWS Lambda with CloudWatch with a section on "How to Use CloudWatch Logs with Lambda". Looks like you already found your answer, but for anybody without the IAM specific issues, this may help.

Upvotes: 0

Sherry Li

Reputation: 1

CloudWatch & CloudWatch Logs are different Permissions, you need add CloudWatch Logs to the policy which attached with your role.

Upvotes: 0

RainRain

Reputation: 453

I encountered this problem but none of the answers above solved my issue. It turns out that the region was somehow set to Ohio when I first started CloudWatch. After I changed it to US East (N. Virginia), everything works fine.

Upvotes: 0

kenberkeley

Reputation: 9856

It might already log, we just couldn't find the logs we expect...

e.g.

app.use(function simpleLogger (req, res, next) {
  console.info('[Logger]', req.method, req.originalUrl)
  next()
})

After performing GET /hello?world=1,

Local console: (simple and clear, nice!)

[Logger] GET /hello?world=1

CloudWatch Logs: (can you easily find the exact log below?)

START RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0 Version: $LATEST
2018-12-04T09:26:11.236Z  a3552c34-f7a6-11e8-90ba-2fb886f31fb0  [Logger] GET /hello?world=1
END RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0
REPORT RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0  Duration: 41.02 ms  Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 29 MB

Conclusion: too verbose to locate the original logs.

Upvotes: 0

Aniket Thakur

Reputation: 68985

As other answers state you need to give lambda permission to post logs to cloud watch logs. AWS had provided AWSLambdaExecute policy just for that. It's json is -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::*"
        }
    ]
}

You can add this policy in your role which is assigned to your lambda and you should start seeing the logs.

NOTE: It also has S3 read/write access. If you do not want it you can create a custom policy with just the logs part.

Upvotes: 5

Vic

Reputation: 662

Apparently another necessity for logging to happen is the Lambda function must indicate completion; for instance in the Python context, the handler must return something other than None.

Upvotes: 9

Anupam Mahapatra

Reputation: 833

For the issue was I was trying to create a log group in the Cloudformation script by : AWS::Logs::LogGroup and then trying to push the Lambda log to this log group. :P Novice After careful reading , i found that Lambda creates its own log with the aforementioned format: /aws/lambda/ We just need to provide policy permission to this log group , or just a generic permission with resource as: arn:aws:logs:::*

hope this helps

Upvotes: 1

Clive Sargeant

Reputation: 652

Maybe a bit late, but for those who still struggle with seeing the lambda logs in cloudwatch. I noticed this regarding the lambda function's execution role: "You may use an existing role with this function. Note that the role must be assumable by Lambda and must have Cloudwatch Logs permissions." So in IAM i granted " CloudWatchLogsFullAccess" to the role i assigned to my function. then in cloudwatch, under logs, you'll see the logs for the functions assigned this role.

Upvotes: 0

tomusiaka

Reputation: 4141

For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions

I already had these permissions yet it did not work.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

I changed the timeout, saved and logs still did not work.

I assigned another role and logs still did not work.

What ended up working for me was clicking "Create a custom role", then "Allow". This was it and logs started being generated but since I did not want to use a new role but my existing role, I simply assigned my existing role afterwards and it worked. So technically I should have returned back to original configuration that did not work but now it works. Go figure.

Upvotes: 22

Leonardo Max Almeida

Reputation: 46

Make sure you have the full path of your "Existing role" in your lambda function "Configuration":

Role: Choose an existing role Existing Role: service-role/yourRoleName

For some reason, typing only yourRoleName will work for some services (like SES) but not for CloudWatch.

Also, you may try creating a new role instead of using an existing one. This will create the role with the proper configuration (hopefully).

Upvotes: 1

Can&#39;t get AWS Lambda function to log (text output) to CloudWatch

Answers (18)

July 2020 Update !!

e.g.

Related Questions

Can't get AWS Lambda function to log (text output) to CloudWatch