CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
A A
A A

Reputation: 61

Cannot delete Amazon S3 key that contains bad character

I just began to use S3 recently. I accidentally made a key that contains a bad character, and now I can't list the contents of that folder, nor delete that bad key. (I've since added checks to make sure I don't do this again).

I was using an old "S3" python module from 2008 originally. Now I've switched to boto-2.0, and I still cannot delete it. I did quite a bit of research online, and it seems the problem is I have an invalid XML character, so it seems a problem at the lowest level, and no API has helped so far.

I finally contacted Amazon, and they said to use "s3-curl.pl" from http://aws.amazon.com/code/128. I downloaded it, and here's my key:

<Key>info/&#x1b;[01</Key>

I think I was doing a quick bash for loop over some files at the time, and I have "lscolors" set up, and so this happened.

I tried ./s3curl.pl --id <myID> --key <myKEY> -- -X DELETE https://mybucket.s3.amazonaws.com/info/&#x1b;[01

(and also tried putting the URL in single/double quotes, and also tried to escape the '[').

Without quotes on the URL, it hangs. With quotes, I get "curl: (3) [globbing] error: bad range specification after pos 50". I edited the s3-curl.pl to do curl --globoff and still get this error.

I would appreciate any help.

Upvotes: 3

Views: 3134

Answers (5)

Lucas Ocon
Lucas Ocon

Reputation: 199

This solved the issue, just delete the main folder:

aws s3 rm "s3://BUCKET_NAME/folder/folder" --recursive

Upvotes: 5

Aqsa javed
Aqsa javed

Reputation: 319

I recently encountered this case. I had newline at the end of my bucket. The following command solved the matter.

 aws s3 rm "bucket_name"$'\r' --recursive

Upvotes: 0

gordy
gordy

Reputation: 9786

I was in this situation recently, to list the items you can use:

aws s3api list-objects-v2 --bucket my_bucket --encoding-type url

the bad keys will come back url encoded like:

"Key": "%01%C3%B4%C2%B3%C3%8Bu%C2%A5%27%40yr%3E%60%0EQ%14%C3%A5.gif"

spaces became + and I had to change those to %20 and * wasn't encoded I had to replace those with %2A before I was able to delete them.

To actually delete them, I wasn't able to use the aws cli because it would urlencode the already urlencoded key resulting in a 404, so to get around that I manually hit the rest API with the DELETE verb.

Upvotes: 0

hansaplast
hansaplast

Reputation: 11573

In my case there were newlines in the key (however that happened..). I was able to fix it with the aws cli like this:

aws cli rm "s3://my_bucket/Icon"$'\r'

I also had versioning enabled, so I also needed to do this, for all the versions (versions ids are visible in the UI when enabling the version view):

aws s3api delete-object --bucket my_bucket --key "Icon"$'\r' --version-id <version_id>

Upvotes: 2

allg00d
allg00d

Reputation: 41

You can use the s3cmd tool from here. You first need to run

s3cmd fixbucket <bucket name that contains bad file>.

You can then delete the file using

s3cmd del <bucket>/<file>

Upvotes: 4

Related Questions