Looking for usage details on Amazon S3

Question

I'm trying to look for reports on my Amazon S3 usage, but Amazon only provides simple summary on the usage, such as amount of storage / transfer in a particular month. I need to have a breakdown of these data by files, for example:

• abc.mp3 : 123 GET request / 0.12Mb transferred
• hello.mp4 : 345 GET request / 0.32Mb transferred
• fun.docx : 834 GET request / 0.20Mb transferred

Also, I need to know where do these GET request coming from, so I can better monitor and control on the S3 usage. For example:

abc.mp3:

• 53 GET request from http://www.example.com/music/page1.html
• 70 GET request from http://www.example.com/music/page2.html

Any tools / method on achieving these? Thanks!

Answer 1

You need to enable S3 access logs: http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html

Then you should be able to parse the logs for the information you want. Once you start getting logs there are many options for parsing them, here are a few that I found with a quick search:

s3stat

s3-logs-analyzer

Loggly S3 support