CODEWITHSUNDEEP
aws-lambda
user373201
user373201

Reputation: 11435

redirect aws lambda logs to a particular log group in cloudwatch

I have multiple lambda's. Is there a way to direct the logs from all these lambda to a specific cloud watch log group instead of each going to their own.

Upvotes: 3

Views: 2859

Answers (2)

FG_online555
FG_online555

Reputation: 9

What you can do is to set up a log group and make sure all lambdas log into that one log group. Unfortunately, AWS will still log to the default log group of the lamdba "aws/lambda/", but you will also have the one log group with all the logs aggregated - by creating a logger using aws_lambda_powertools

local_logger = Logger(
service="name", level=os.environ.get("LOG_LEVEL", "INFO").upper(), region="region"

))

and replacing it with a logger that logs to your log_group:

logs_client = boto3.client('logs', region_name=os.environ['AWS_REGION'])
# Send logs to the custom log stream
logs_client.create_log_stream(
    logGroupName=log_group_name,
    logStreamName=log_stream_name
)
# Add the custom handler to the logger
local_logger.addHandler(CloudWatchLogHandler(logs_client, log_group_name, log_stream_name))

import logging

from aws_lambda_powertools import Logger


class CloudWatchLogHandler(logging.StreamHandler):

    def __init__(self, log_client: Logger, log_group_name: str, log_stream_name: str, context_id: str):
        super().__init__()
        self.logs_client = log_client
        self.log_group_name = log_group_name
        self.log_stream_name = log_stream_name
        self.context_id = context_id

    def emit(self, record):

        log_entry = self.format(record)

        log_json = {
            'level': record.levelname,
            'location': f"{record.funcName}:{record.lineno}",
            'message': log_entry,
            'timestamp': int(record.created * 1000),
            'service': record.name,
            'correlation_id': self.context_id
        }

        self.logs_client.put_log_events(
            logGroupName=self.log_group_name,
            logStreamName=self.log_stream_name,
            logEvents=[{
                'timestamp': int(record.created * 1000),
                'message': str(log_json),
            }]
        )

Upvotes: 0

user373201
user373201

Reputation: 11435

Turns out the other question is similar and looks like there is no way to currently aggregate logs from different lambda services.

I ended up creating an SQS FIFO queue. Send log messages to the queue from lambda and created a Log Lambda that will basically print all messages from the sqs queue.

When I want to view the logs I go the cloudwatch logs for the Log Lambda which have everything ordered.

Upvotes: 3

Related Questions