CODEWITHSUNDEEP
rubyamazon-web-servicesamazon-s3aws-sdk
Leem.fin
Leem.fin

Reputation: 42652

Listen to object change in AWS S3 (AWS SDK)

In AWS S3, I have a bucket named 'my-bucket', under my-bucket I have the following structure:

my-bucket/
    customers/
         products/
              - data1.txt
              - data2.txt

with AWS Ruby SDK, I can list all the items under my-bucket by ruby code:

require 'aws-sdk'

s3 = Aws::S3::Resource.new(region: 'us-west-2')

bucket = s3.bucket('my-bucket')

# Show only the first 50 items
bucket.objects.limit(50).each do |item|
  puts "Name:  #{item.key}"
end

This is fine, I am wondering is it possible to have a listener listen to any new file object uploaded to my-bucket/customers/products/ with AWS Ruby SDK? If possible, how to do it?

Upvotes: 5

Views: 2123

Answers (1)

johni
johni

Reputation: 5578

It cannot be done directly with code. What you can do, however, is have a listener on a SQS queue (that you can do with code) and create an event in S3 for pushing a message to a queue on every change that is done in your bucket.

For SQS documentation please follow here (it is very straightforward).

For configuring an event you can follow this detailed link, and for a shorter version of this, do as follows (create a SQS queue first):

  1. Go to your bucket (through the AWS Console).
  2. Click the Properties tab (on the upper-left corner).
  3. Expand Events.
  4. Click Add notification.
  5. Choose the type of events you'd like to listen to.
  6. Optional, if you want the event to apply on objects with a specific prefix, specify the prefix.
  7. Optional,, same for suffix.
  8. Send it to SQS Queue.
  9. Put the queue's ARN.

Now, all you need to do with Ruby, is listen (with a polling mechanism) to new message on the queue. The messages will have some JSON format in some schema, play with it.

That should solve your problem.

Upvotes: 3

Related Questions