CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-cloudwatchlogs
manu thankachan
manu thankachan

Reputation: 473

How to Import Logs From An Amazon S3 Bucket to cloudwatch

I have Exported the aws cloudwatch Log Data to Amazon S3 Using the AWS CLI. Followed kb: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html.

Any idea how we can import the logs saved in s3 for analysing the data?

Upvotes: 18

Views: 17611

Answers (2)

andybkay
andybkay

Reputation: 81

The only way to do this is to use a script of some form to pull the files from S3, and put them into CloudWatch Logs. Lambda would be a good candidate, using an S3 trigger if you wanted to automate it.

But, the better way would be to archive the logs. In a DR scenario as long as your log entries are safe, you're ok. Trying to replay those logs into a new cluster sounds like a bad idea to me.

Upvotes: 4

kldnz
kldnz

Reputation: 86

Well, for data analysing I would recommend using Athena,

Regarding your comments a quick search was not able to provide an answer on how to import logs.

Logs are attached to resources that you spin up, so it sounds kind of impossible to attach logs from a newly generated cluster to the old cluster when it is up, since they are two separate resources with different unique id's etc.

Logs are stored in cloudwatch don't have an expiry date even if the resource is deleted. So you would still be able to check the logs of the newly generated cluster when it is no longer online (destroyed) export the logs to s3 and analyse them using Athena.

Upvotes: 3

Related Questions