CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
mye
mye

Reputation: 103

Easy way to created dated subdirectories on AWS S3

I'm trying to create a web service that is able to store user-upload files in S3. The problem is that we want the files stored in "dated directories".

For example, if a user uploads a.txt on 12/1/2017 at 9:15am, the file should look like this in S3:

https://s3-eu-west-1.amazonaws.com/test-bucket/uploaded/2017/12/1/9/a.txt

Does S3 have any API to help us achieving this or do we need to hand-craft this solution?

Upvotes: 0

Views: 46

Answers (2)

John Rotenstein
John Rotenstein

Reputation: 269951

There is no such API in S3. Think of Amazon S3 as a storage service, not an application or database.

It is the responsibility of your application to store the data in the desired naming format -- just like storing data on a disk.

By the way, your naming format could do with some improvement:

  • Always expand fields to the correct number of digits (use 01 for January rather than 1) so that they sort correctly.
  • Think about your use-case -- if you will be scanning documents by year, then the /2017/12/01/09/a.txt naming format makes sense since you can look in the 2017 directory (not that directories really exist in S3). If not, then simply store it as /2017-12-01-09-a.txt.
  • Make it very clear which one is month vs day -- the USA is the only country in the world that treats "12/1/2017" as December 1st. The rest of the world reads it as "12 January". Using the format of 2017-12-01 makes it clear that it is 1-December-2017.
  • What about naming conflicts? Can only one person upload a file with a given name on a given day? How are you going to differentiate between different users uploading a file with the same name?
  • The reality is, the filename is totally irrelevant -- your application should use a database to keep track of objects that users upload and assign each of them a unique name. When a file is later requested, lookup the filename in the database and then provide that file. Do not use S3 filenames as a pseudo-database where the name conveys particular meaning, otherwise you'll often have to rename files to add more meaning!
    • Directories don't actually exist in S3 -- they are just part of the filename. So, you can create a file in a given directory just by storing it -- there is no need to pre-create directories.

Upvotes: 3

Marcel Böttcher
Marcel Böttcher

Reputation: 185

AWS S3 does not provide you with such logic. But it should by fairly easy to use the time information of your application to create such a s3 object key ("path").

Good luck!

Upvotes: 0

Related Questions