Reputation: 173
After some time using AWS I find my cloudwatch is full of obsolete logs, is there a way to delete all log groups and streams by date? So I can clear all previous year logs.
Upvotes: 9
Views: 18003
Reputation: 1021
Logs are kept indefinitely by default.
How ever you can set this by log group:
Log Retention—By default, logs are kept indefinitely and never expire. You can adjust the retention policy for each log group, keeping the indefinite retention, or choosing a retention periods between 10 years and one day.
More Info here: https://docs.aws.amazon.com/en_en/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html
How to change log data retention: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Working-with-log-groups-and-streams.html
Upvotes: 4
Reputation: 116
While you can set the retention period on a stream as @DominikHelps said, that just deletes the messages; it will not delete the stream itself.
You can use the CLI to find the streams for a log group, along with the time they're created:
aws logs describe-log-streams --log-group-name Example --output text --query 'logStreams[*].[creationTime,logStreamName]' | sort -rn
This gives you output that looks like this:
1544374120302 stream1
1544373223032 stream2
1544365017774 stream3
The numbers are epoch timestamps, in milliseconds. You can use a tool like this to translate them into human-readable timestamps. Then, once you've identified the stream(s) that you want to delete, you can again use the CLI:
aws logs delete-log-stream --log-group-name Example --log-stream-name stream1
It's a fairly easy step from doing this manually to doing it as a cronjob. And if you don't mind programming, turning it into a Lambda that's invoked by a CloudWatch Events scheduled event.
Upvotes: 10