Are there monitor tools for AWS S3 and CloudFront

Question

I am using the amazon services S3 and CloudFront for a web application and I would like to have various statistics about accessing the data that I am providing through the logs of those services (there is logging activated in both services).

I did a bit of googling and the only thing I could find is how to manage my S3 storage. I also noticed that newrelic offers monitoring for many amazon services but not for those 2.

Is there something that you use? A service that could read my logs periodically and provide me with some nice analytics that would make developers and managers happy?

e.g. I am trying to avoid writing my own log parsers.

Answer 1

Popular 3rd party analytics packages include S3stat, Cloudlytics and Qloudstat. They all run around $10/month for low traffic sites.

Several stand-alone analytics packages support Amazon's logfile format if you want to download logs each night and feed them in directly. Others might need pre-processing to transform to Combined Logfile Format (CLF) first.

I've written about how to do that here:

https://www.expatsoftware.com/articles/2007/11/roll-your-own-web-stats-for-amazon-s3.html

Answer 2

I might be answering this very late. But I have worked on a golang library that can run analysis of CDN and S3 usages and store them in a backend of your choice varying from influxdb, MongoDB or Cassandra for later time series evaluations. The project is hosted at http://github.com/meson10/cdnlysis

See if this fits.

Answer 3

I believe Piwik supports the Amazon S3 log format. Take a look at their demo site to see some example reports.

Answer 4

Well, we have a SaaS product Cloudlytics which offers you many reports including, Geo, IP tracking, SPAM, CloudFront cost analysis. You can try it for free for upto 25 MB of logs.

Answer 5

Well, this may not be what you expect but I use qloudstat for my cloudfront distributions.

The $5 plan covers my needs thats less than a burrito here where I live.

Best regards.