CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
porton
porton

Reputation: 5827

Backing up to Amazon and versioning

I was assigned the task to backup our data from Google Storage to Amazon AWS.

Could you explain which tool is the best for this task? (We can use a free Linux tool.)

I also think of using versioning in Amazon for the case if a file is overwritten. Because of this, I also ask what happens if an object on a versioned Amazon store is overwritten with new version with the same object content (the same file)? If an object is overwritten with new version with the same object content, is new version created or no?

Upvotes: 0

Views: 35

Answers (1)

Michael - sqlbot
Michael - sqlbot

Reputation: 179404

If you are synchronizing data, you need to predict and avoid transferring duplicate content. Otherwise, you're going to pay unnecessary bandwidth charges.

The S3 List Objects API v1 and v2 and the List Object Versions APIs all provide you with size, timestamp, and etag, among other attributes, in chunks of 1000 keys or key versions per request (paginated). The etag is the MD5 of the body for single-part uploads < 5 GB. The etag is the MD5 of the concatenated binary MD5s of the parts, for multipart uploads.

If you overwrite a versioned object in S3 with an entirely identical object, S3 retains the old version and stores the new version. The new version becomes the current version. Both are retained and both are billed for storage.

Could you explain which tool is the best

No, we don't do that here.

Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam.

Upvotes: 1

Related Questions