CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-cloudwatch
pnocti
pnocti

Reputation: 97

upload custom logs in s3 to cloudwatch for metrics monitoring

I created a custom app that automatically uploads logs to s3. Is there a way to push those logs to cloudwatch from s3 for analysis and alerting? I'm aware that I can use a cloudwatch agent to push directly to cloudwatch from the app but there are complications involved in that option. Thank you!

Upvotes: 0

Views: 1461

Answers (2)

j_sve
j_sve

Reputation: 29

You could probably use Cloudwatch Events to listen to S3 changes. Not sure about if you can get the data from the S3 file, or just a trigger saying that a new log has been added.

You could also use S3 event notifications (https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html) connected either to a lambda or SQS, and from there create the logs to Cloudwatch. (similar to what was suggested by @marcin)

A better solution, but one that is a bit beyond the scope of the question, would be to send your logs through Kinesis Firehose and from there add the Cloudwatch and S3 logs.

Upvotes: 1

Marcin
Marcin

Reputation: 238051

I'm not aware of any out-of-the-box mechanism for that provided by AWS. But I think it could be relatively easy to develop.

Namely, you can create S3 notification for a PUT of a new log file from your app to S3. The event would trigger a lambda function. The function would get the file and using AWS SDK, e.g. boto3's put_log_events, it would send the log events to CloudWatch logs.

Upvotes: 0

Related Questions