CODEWITHSUNDEEP
amazon-web-servicesloggingaws-lambdaaws-cli
skybunk
skybunk

Reputation: 863

AWS Lambda Logs using AWSCLI: How can I access Lambda logs using AWSCLI?

I was trying to work with AWS Lambda using the awscli on an ubuntu ec2 instance, and I do not have access to the aws console. Note that I am not using serverless or zapper, I directly zip my main.py file along with the dependency files as mentioned here

I run the function like this

aws lambda invoke --function-name python-test --invocation-type RequestResponse outfile.txt

The errors given in the outfile are very vague and dont help in debugging, rather they confuse me more. Using the admin's system, I am able to recognize the errors when I run a test on the console, but How can I check those logs using the awscli ?

So I tried running aws cloudwatch list-metrics > cloudwatch_logs.log and searching the function name 'python-test' in the cloudwatch_logs.log file, I am able to find Namespace, MetricName, Dimensions for this function, but how do you access the logs ?

Any help with links to similar examples greatly appreciated !

Upvotes: 28

Views: 17145

Answers (5)

Noan Cloarec
Noan Cloarec

Reputation: 2365

Here is a script I wrote that automates John Rotenstein's answer using jq

#!/bin/bash

# Usage: ./lambdalogs.sh my-function-name or ./lambdalogs.sh part-of-my-function-name

lambda_regex=$1

log_group=$(aws logs describe-log-groups --query logGroups[*].logGroupName | jq -r --arg pattern "aws/lambda/.*$lambda_regex.*" '.[] | select(. | test($pattern))' | head -n 1)
echo "Using log group $log_group"

log_stream=$(aws logs describe-log-streams --log-group-name $log_group --query 'logStreams[*].logStreamName' --order-by LastEventTime --descend | jq -r '.[0]')
echo "Latest log stream is $log_stream"

aws logs get-log-events --log-group-name $log_group --log-stream-name $log_stream | jq -r '.events | .[] | .timestamp |= (. / 1000 | strftime("%Y-%m-%d %H:%M:%S")) | .timestamp, .message'

Upvotes: 0

Vincent Claes
Vincent Claes

Reputation: 4768

This one liner invokes the lambda function and prints the logs in your terminal:

aws lambda invoke --function-name my-lambda-function --cli-binary-format raw-in-base64-out --payload '{"foo":"bar"}' output.json --log-type Tail --query 'LogResult' --output text | base64 -d

Upvotes: 4

chachan
chachan

Reputation: 2452

jq flavor:

List AWS lambda group name (if list is too big you might want to filter it with grep):

aws logs describe-log-groups | jq -r ".logGroups[].logGroupName"

Then read message property from latest stream with:

LOG_GROUP_NAME="/aws/lambda/awesomeFunction"
LOG_STREAM_NAME=$(aws logs describe-log-streams --log-group-name "${LOG_GROUP_NAME}" | jq -r '.logStreams | sort_by(.creationTime) | .[-1].logStreamName')
aws logs get-log-events --log-group-name "${LOG_GROUP_NAME}" --log-stream-name "${LOG_STREAM_NAME}" | jq -r '.events[] | select(has("message")) | .message'

You might want to put this in a logs.sh file.

If you want more or other streams you might want to tweak the .logStreams[0] part

Upvotes: 10

a-h
a-h

Reputation: 4284

Using the AWS CLI can be a bit irritating because the stream name changes as you modify your function. I've found that using awslogs (https://github.com/jorgebastida/awslogs) is a nicer workflow.

List the groups:

awslogs groups

Filter results.

awslogs groups|grep myfunction

Then get the logs from the group.

awslogs get /aws/lambda/ShortenStack-mutationShortenLambdaBC1758AD-6KW0KAD3TYVE

It defaults to the last 5 minutes, but you can add the -s parameter to choose a time, e.g -s 10m for the last 10 minutes.

The output is colourised if you're at the terminal, or plain if you're piping it through other commands, e.g. grep to find something.

Upvotes: 2

John Rotenstein
John Rotenstein

Reputation: 269826

First, get the log group name:

aws logs describe-log-groups --query logGroups[*].logGroupName
[
    "/aws/lambda/MyFunction"
]

Then, list the log streams for that log group:

aws logs describe-log-streams --log-group-name '/aws/lambda/MyFunction' --query logStreams[*].logStreamName

[
    "2018/02/07/[$LATEST]140c61ffd59442b7b8405dc91d708fdc"
]

Finally, get the log events for that stream:

aws logs get-log-events --log-group-name '/aws/lambda/MyFunction' --log-stream-name '2018/02/07/[$LATEST]140c61ffd59442b7b8405dc91d708fdc'

{
    "nextForwardToken": "f/33851760153448034063427449515194237355552440866456338433", 
    "events": [
        {
            "ingestionTime": 1517965421523, 
            "timestamp": 1517965421526, 
            "message": "START RequestId: bca9c478-0ba2-11e8-81db-4bccfc644168 Version: $LATEST\n"
        }, 
        {
            "ingestionTime": 1517965424581, 
            "timestamp": 1517965424567, 
            "message": "END RequestId: bca9c478-0ba2-11e8-81db-4bccfc644168\n"
        }, 
        {
            "ingestionTime": 1517965424581, 
            "timestamp": 1517965424567, 
            "message": "REPORT RequestId: bca9c478-0ba2-11e8-81db-4bccfc644168\tDuration: 3055.39 ms\tBilled Duration: 3100 ms \tMemory Size: 128 MB\tMax Memory Used: 35 MB\t\n"
        }
    ], 
    "nextBackwardToken": "b/33851760085631457914695824538087252860391482425578356736"
}

Upvotes: 74

Related Questions