CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
Software Ninja
Software Ninja

Reputation: 83

Organizing files in S3

I have a social media web application. Users upload pictures such as profile picture, project pictures, and etc. What's the best way to organize these files in a S3 bucket?

I thought of creating a folder with userid as its name inside the bucket and the inside that multiple other folders i.e. profile, projects and etc.

Not sure if that's the best approach to follow!

Upvotes: 4

Views: 3353

Answers (2)

Arafat Nalkhande
Arafat Nalkhande

Reputation: 11708

Earlier there used to be a need to add random prefixes for better performance. More details here and here.

Following is the extract from one of that pages

Pay Attention to Your Naming Scheme If:

Distributing the Key names

Don’t save your object's key name starts with a date or standard key names, it improves complexity in the S3 indexing and will reduce performance, because based on the indexing objects saves in the single storage partition .

Amazon S3 maintains keys lexicographically in its internal indices.

However, as of 17 Jul 2018 announcement, adding random prefix to S3 key isn't required for improving the performance

Upvotes: 2

John Rotenstein
John Rotenstein

Reputation: 269480

The names (Keys) you assign an object in Amazon S3 are frankly irrelevant.

What matters is that you have a database that tracks the objects, their ownership and their purpose.

You should not use the filename (Key) of an Amazon S3 object as a way of storing information about the object, because your application might have millions of objects in S3 and it is too slow to scan the list of objects to see which ones exist. Instead, consult a database to find them.

To answer your question: Yes, create a prefix by username if you wish, but then just give it a unique name (eg a Universally unique identifier - Wikipedia) that avoids name clashes.

Upvotes: 5

Related Questions