CODEWITHSUNDEEP
amazon-web-servicesruby-on-rails-3amazon-s3wsdl
baash05
baash05

Reputation: 4516

Get a list of files on S3 that have changed since X

I have about 40,000 images up on S3 and I've downloaded into my application/database then sent them out to another site (like ebay or magento)
This is to support a client that sells his products on a few sites. Sites which really you'd rather keep a copy of the product image on their site. (so they can resize it and such)

My issue right now is that I want to poke S3 every once and a while looking for new files, or modified files.
I don't much like the idea of targeting each file one at a time. Nor do I like the idea of bringing down all the file names and dates and then comparing them with dates I've stored. Both seem to be quite wasteful especially if I want to run this every day (or every hour).

What I had hoped for, and what I'm looking for is a way to say "give me the names of all the files that have changed since 2013-10-14 13:10:30. This would let me store just one value, and if nothings changed, then I'd get back nothing (or something that indicated nothing).

Is there a way to get a list of changed files since X date?

I'm language agnostic.. though Ruby/Rails would be cool.

Note: I've tried to figure it out with the WSDL, but it doesn't quite seem to help as much as I'd hope.

Upvotes: 1

Views: 501

Answers (1)

dcro
dcro

Reputation: 13679

Unfortunately S3 does not offer any support for this.

Currently your only options are to either list all the objects in the S3 bucket and check for changes or to keep track of the changed objects separate from S3 (record the last changed timestamp in some data store when you change the objects).

Upvotes: 1

Related Questions