CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
Chaitanya Srivastava
Chaitanya Srivastava

Reputation: 83

Recover Deleted Objects From Amazon S3

I have a bucket (version enabled), how can i get back the objects that are accidentally permanent deleted from my bucket.

Upvotes: 6

Views: 16575

Answers (5)

Maximus
Maximus

Reputation: 1547

This version of the script worked really well for me. I have a bucket that has a directory with 180,000 items in it, and this one chews through them and restores all the files that are in a directory/folder that is within the bucket.

If you just need to restore all the items in a bucket that don't have a directory, then you can just drop the prefix parameter.

#!/bin/bash

BUCKET=mybucketname
DIRECTORY=myfoldername

function run() {

  aws s3api list-object-versions --bucket ${BUCKET_NAME} --prefix="${DIRECTORY}" --query='{Objects: DeleteMarkers[].{Key:Key}}' --output text |
  while read KEY
  do
    if [[  "$KEY" == "None" ]]; then
      continue
    else
      KEY=$(echo ${KEY} | awk '{$1=""; print $0}' | sed "s/^ *//g")  
      VERSION=$(aws s3api list-object-versions --bucket ${BUCKET_NAME} --prefix="$KEY" --query='{Objects: DeleteMarkers[].{VersionId:VersionId}}' --output text | awk '{$1=""; print $0}' | sed "s/^ *//g") 
      echo ${KEY} 
      echo ${VERSION}
    fi
    aws s3api delete-object --bucket ${BUCKET_NAME} --key="${KEY}" --version-id ${VERSION}
  done

}

Note, running this script two times will run, but it won't work. It will just return the same record in the second script, so it doesn't really do anything. If you had a massive bucket, I might setup 3-4 scripts that filter by files that start with a certain letter/number. At least this way you can start working on files deeper down in the bucket.

Upvotes: 0

Gonzalo Moreno Caballero
Gonzalo Moreno Caballero

Reputation: 398

If your bucket objects has white spaces in filename, previous scripts may not work properly. This script take the key including white spaces.

#!/bin/bash
#please provide the bucketname and path to destination folder to restore
# Remove all versions and delete markers for each object
aws s3api list-object-versions --bucket $1 --prefix $2 --output text |
grep "DELETEMARKERS" | while read obj
  do
    KEY=$( echo $obj| awk '{indice=index($0,$(NF-1))-index($0,$3);print substr($0, index($0,$3), indice-1)}')
    VERSION_ID=$( echo $obj | awk '{print $NF}')
    echo $KEY
    echo $VERSION_ID
    aws s3api delete-object --bucket $1 --key "$KEY" --version-id $VERSION_ID
  done

Upvotes: 3

Joshua Young
Joshua Young

Reputation: 61

Thank you, Kc Bickey, this script works wonderfully! Only thing I might add for others is to make sure " $VERSION_ID" immediately follows "--version-id" on line 12. The forum seems to have wrapped " $VERSION_ID" to the next line and it causes the script to error until that's corrected.

**Script:**
#!/bin/bash
#please provide the bucketname and path to destination folder to restore
# Remove all versions and delete markers for each object
 aws s3api list-object-versions --bucket $1 --prefix $2 --output text | 
 grep "DELETEMARKERS" | while read obj
   do
        KEY=$( echo $obj| awk '{print $3}')
        VERSION_ID=$( echo $obj | awk '{print $5}')
        echo $KEY
        echo $VERSION_ID
        aws s3api delete-object --bucket $1 --key $KEY --version-id $VERSION_ID

   done

Upvotes: 6

Kc Bickey
Kc Bickey

Reputation: 1286

I have created a script to restore the objects with deletemarker. You'll have to input it like below:

sh Undelete_deletemarker.sh bucketname path/to/certain/folder

**Script:**
#!/bin/bash
#please provide the bucketname and path to destination folder to restore
# Remove all versions and delete markers for each object
 aws s3api list-object-versions --bucket $1 --prefix $2 --output text | 
 grep "DELETEMARKERS" | while read obj
   do
        KEY=$( echo $obj| awk '{print $3}')
        VERSION_ID=$( echo $obj | awk '{print $5}')
        echo $KEY
        echo $VERSION_ID
        aws s3api delete-object --bucket $1 --key $KEY --version-id 
        $VERSION_ID

   done

Happy Coding! ;)

Upvotes: 7

Frederic Henri
Frederic Henri

Reputation: 53703

with bucket versioning enable to permanently delete an object you need to specifically mention the version of the object DELETE Object versionId If you've done so you cannot recover this specific version, you get access to previous version

When versioning is enabled, a simple DELETE cannot permanently delete an object. Instead, Amazon S3 inserts a delete marker in the bucket so you can recover from this specific marker, but if the marker is deleted (and you mention it was permanent deleted) you cannot recover

did you enable Cross-Region Replication ? If so you can retrieve the object in the other region:

If a DELETE request specifies a particular object version ID to delete, Amazon S3 will delete that object version in the source bucket, but it will not replicate the deletion in the destination bucket (in other words, it will not delete the same object version from the destination bucket). This behavior protects data from malicious deletions.

Edit: If you have versioning enabled on your bucket you should get the Versions Hide/Show toggle button and when Show is selected you should have the additional Version ID column as per the screenshot from my bucket S3 Management Console

Upvotes: 2

Related Questions