CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
Darlyn
Darlyn

Reputation: 4938

AWS S3 Deleting file while someone is downloading that object

I cant seem to find how does AWS s3 handles, if someone deletes file while other person is downloading it. Does it behave like unix system, where descriptor is opened and file is downloaded without problems or does it behave in other way?

Thanks for help!

Upvotes: 3

Views: 1689

Answers (2)

jarmod
jarmod

Reputation: 78603

You will face a race condition, of sorts, so the outcome is unpredictable. When you download a file from S3, you might be connected to multiple S3 servers. If at any time, you request part of an S3 object and the server you are connected to thinks the object has been deleted then your download will fail.

Here's a simple test: store a 2GB flle in S3, then download it. While it it is downloading, go into the S3 console and delete the object. You will find that your download fails (with NoSuchKey) because the specified key no longer exists.

Create a temporary 2GB file and upload it to S3:

mkfile -n 2g 2gb.dat

$ aws s3 cp 2gb.dat s3://mybucket
upload: ./2gb.dat to s3://mybucket/2gb.dat

Once complete, start downloading the file:

$ aws s3 cp s3://mybucket/2gb.dat fred.dat
Completed 162.2 MiB/2.0 GiB (46.9 MiB/s) with 1 file(s) remaining

Then jump over to the S3 console and delete 2gb.dat, and this will happen:

$ aws s3 cp s3://mybucket/2gb.dat fred.dat
download failed: s3://mybucket/2gb.dat to ./fred.dat An error occurred 
(NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Upvotes: 0

franklinsijo
franklinsijo

Reputation: 18270

S3 offers eventual consistency for DELETES.

From the S3 Data Consistency Model

A process deletes an existing object and immediately tries to read it. Until the deletion is fully propagated, Amazon S3 might return the deleted data.

Here, where the deletion and downloading of the same object is done concurrently, even if the deletion of the object succeeds before the download is complete, the process will still be able to download the data.

Upvotes: 2

Related Questions