CODEWITHSUNDEEP
amazon-web-services
user5556585
user5556585

Reputation: 213

Cloud watch logs prepending timestamp to each line

We have cloud watch log agent setup and the logs streamed are appending a timestamp to beginning of each line which we could see after export.

2017-05-23T04:36:02.473Z "message"

Is there any configuration on cloud watch log agent setup that helps not appending this timestamp to each log entry? Is there a way to export cloud watch logs only the messages of log events? We dont want the timestamp on our exported logs.

Thanks

Upvotes: 6

Views: 2238

Answers (2)

Montaro
Montaro

Reputation: 9848

I don't think it's possible, I needed the same exact behavior you are asking for and looks like it's not possible unless you implement a man in the middle processor to remove the timestamp from every log message as suggested in the other answer

Checking the CloudWatch Logs Client API in the first place, it's required to send the timestamp with every log message you send to CloudWatch Logs (API reference)

And the export logs to S3 task API also has no parameters to control this behavior (API reference)

Upvotes: 0

Binh Nguyen
Binh Nguyen

Reputation: 2157

Assume that you are able to retrieve those logs using your Lambda function (Python 3.x).

Then you can use Regular Expression to identify the timestamp and write a function to strip it from the event log.

^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z\t

The above will identify the following timestamp: 2019-10-10T22:11:00.123Z

Here is a simple Python function:

def strip(eventLog):
    timestamp = "r'^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z\t'"
    result = re.sub(timestamp, "", eventLog)
    return result

Upvotes: 0

Related Questions