CODEWITHSUNDEEP
djangoamazon-s3python-django-storages
Khandelwal
Khandelwal

Reputation: 661

django-storages and amazon s3 - suspiciousoperation

I'm using django-storages with Amazon S3. I see the following error somewhat intermittently: 

name = self._normalize_name(self._clean_name(name))\n\n  File \"/app/.heroku/venv/lib/python2.7/site-packages/storages/backends/s3boto.py\", line 237, in _normalize_name\n    name)\n\nSuspiciousOperation: Attempted access to 'https:/plantvillage.s3.amazonaws.com/avatar/hans9_avatar.jpg'

Note the single / after https:.

Does anyone know why this shows up? It doesn't happen all the time. I can successfully do this in other cases.

Upvotes: 9

Views: 3108

Answers (5)

Mateus Padua
Mateus Padua

Reputation: 193

I fixed this, adding SuspiciousOperation on except:

class S3CustomStorage(S3BotoStorage):
    def _normalize_name(self, name):
        try:
            return safe_join(self.location, name)
        except (SuspiciousOperation, ValueError):
            return ""

Upvotes: -1

danius
danius

Reputation: 2774

_normalize_name does a lot of fancy and mostly unnecessary on Django stuff with the URL. In my case I just override the S3BotoStorage like this:

class S3CustomStorage(S3BotoStorage):
def _normalize_name(self, name):
    """
    Get rid of this crap: http://stackoverflow.com/questions/12535123/django-storages-and-amazon-s3-suspiciousoperation
    """
    return name

Then use it in the storage property:

ImageField(storage=S3CustomStorage())

And it worked for django simple ImageField with this base configuration:

AWS_ACCESS_KEY_ID = 'TTTT'
AWS_SECRET_ACCESS_KEY = 'XXXX'
AWS_STORAGE_BUCKET_NAME = 'ZZZZ'

Upvotes: 4

manojlds
manojlds

Reputation: 301477

Setting

MEDIA_ROOT=''

fixed the problem for me.

Upvotes: 0

Arctelix
Arctelix

Reputation: 4576

When you use default_storage methods make sure to use the file.name:

Correct:

default_storage.delete(file.name)

Wrong:

default_storage.delete(file.url)

Wrong:

default_storage.delete(file)

All three examples above work with local files, but when using s3 you will run into this error unless you use file.name.

Upvotes: 2

toxotes
toxotes

Reputation: 1464

I haven't gotten S3 storage working on my own project yet, but I did just run across this error, and might be able to point you in the right direction.

If you look at S3BotoStorage._clean_name(), it's just: return os.path.normpath(name).replace('\\', '/'). os.path.normpath() converts the // in your URL to \\, and then .replace() converts that to \. Then, S3BotoStorage._normalize_name() checks to make sure this broken URL is part of the location it represents, which of course it's not. That's where the SuspiciousOperation error is being raised.

So 'name' looks like it's meant to be a local path, instead of the entire AWS URL. In my case, the immediate cause was FILEBROWSER_DIRECTORY = MEDIA_URL + "uploads/" in settings.py, which I had tried hoping to fix a different error about a missing upload folder.

Upvotes: 1

Related Questions