CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
learning_to_write_good_code
learning_to_write_good_code

Reputation: 333

Store list of Strings in S3

I am new to Amazon AWS S3. One of my applications processes 40000 updates an hour with a unique identifier for each update. This identifier is basically a string. At runtime, I want to store the ID in an S3 bucket for all updates.

But, as far as I understood, we need to store files in s3.

Is there anyway around this? Should I store a file.. Then read that file each time..append the name and store it again?

Any direction would be very helpful. Thanks in advance.

I want it to be stored like:

Id1

Id2

Id3 . . , .

Edit: Thanks for the responses, I have added what is asked..

Upvotes: 0

Views: 1069

Answers (1)

ghost
ghost

Reputation: 1217

In S3, you do not have concept of files and folders. All you have is a bucket and objects inside the bucket. However, the UI of AWS groups objects with common prefixes such that they appear to be in the same folder.

Also, there is nothing like appending to a file in S3. Since S3 has objects, what essentially happens is that the so called append deletes the previous object and creates a new object with the previous object's data appended with some more data.

So, one way to do what I think you're trying is :

Suppose you have all the IDs written at 10:00 in an S3 object called data_corresponding_to_10_00_00. For the next hour(and 40000 updates), if they have all new IDs, you can write them to another S3 object with the name data_corresponding_to_11_00_00.

However, if you do not want multiple entries in both the files, and you need to update the previous file itself, using S3 is not a great idea. Rather use a database indexed on ID so that the performance becomes faster.

Upvotes: 1

Related Questions