[Django][AWS S3] botocore.exceptions.clienterror an error occurred (accessdenied) when calling the PutObject operation

Question

I am trying to connect Django project to AWS S3.

settings.py contains below:

AWS_ACCESS_KEY_ID = #ID
AWS_SECRET_ACCESS_KEY = #Key
AWS_STORAGE_BUCKET_NAME = #Bucket
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'backend/static'),
]
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

IAM user is created with AmazonS3FullAccess. But when I enter:

python manage.py collectstatic

an error occurs:

You have requested to collect static files at the destination location as specified in your settings.

This will overwrite existing files! Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/init.py", line 381, in execute_from_command_line utility.execute() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/init.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/base.py", line 316, in run_from_argv self.execute(*args, **cmd_options) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/base.py", line 353, in execute output = self.handle(*args, **options) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 188, in handle collected = self.collect() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 114, in collect handler(path, prefixed_path, storage) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 353, in copy_file self.storage.save(prefixed_path, source_file) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/files/storage.py", line 49, in save return self._save(name, content) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 506, in _save self._save_content(obj, content, parameters=parameters) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 521, in _save_content obj.upload_fileobj(content, ExtraArgs=put_parameters) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/boto3/s3/inject.py", line 621, in object_upload_fileobj ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/boto3/s3/inject.py", line 539, in upload_fileobj return future.result() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/futures.py", line 106, in result return self._coordinator.result() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/futures.py", line 265, in result raise self._exception File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/tasks.py", line 126, in call return self._execute_main(kwargs) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/tasks.py", line 150, in _execute_main return_value = self._main(**kwargs) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/upload.py", line 692, in _main client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

So, I edited bucket policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow All",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::<bucket name>/*"
        }
    ]
}

but the error still occurred. How can I resolve this error?

A tutorial that I am following doesn't show any error at this step.(https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html)

Answer 1

For me the issue was wrong environment variables.

# settings file
AWS_S3_ACCESS_KEY_ID = os.environ["AWS_S3_ACCESS_KEY_ID"]
AWS_S3_SECRET_ACCESS_KEY = os.environ["AWS_S3_SECRET_ACCESS_KEY"]

AWS_S3_ACCESS_KEY_ID="random-key-id"
AWS_S3_SECRET_ACCESS_KEY="random-access-key"

After I removed the double quotes I ran the command again. It works now.

Answer 2

If you are still experiencing these difficulties, the issue lies with the AWS S3 bucket. You can resolve the problem by enabling Access Control List (ACL) on the S3 bucket. Follow the steps below to make the necessary changes:

1. Navigate to your S3 Bucket and click on the "Permissions" tab.
2. Scroll down to the section labeled "Object Ownership" and select the "Edit" option.
3. Modify the settings from "ACL disabled" to "ACL enabled" and save the changes.

Answer 3

If anyone is still having these issues, The problem is on the AWS S£ bucket and You can fix the problem by enabling ACL on the s3 bucket. To do that,

1. Go to your S3 Bucket>Permissions Tab
2. Scroll down to Object Ownership and click on Edit
3. Change the settings from ACL disabled to ACL enabled and save changes

Answer 4

It is Access Control List(ACL) Buckets -> Permission -> ACL -> Edit -> tick Everyone(public access) List and Read for Objects and bucket ACL

Answer 5

Setting AWS_S3_REGION_NAME='your-region' eg: 'us-east-2'

Answer 6

It was AWS S3 access problem.

In S3 bucket console, I edited bucket's public access as public.

NB : Only do this if your intention is to make the file publicly available for example of you're using it to serve files for your website, like images, css etc things that everyone needs to have access to.

Answer 7

This worked for me:

In my S3 bucket -> Permissions Tab -> click Block public access -> Edit -> untick Block all public access -> Save

AND

In my AWS IAM settings -> Users Tab (under Access Management) -> <my-user> -> Add Permissions -> add AmazonS3FullAccess

This granted the user (identified by AWS id and AWS secret) access to control my s3 buckets

Answer 8

By default when you create a new bucket all the public access of s3 objects are blocked(it is ticked by default). that is,you can not access the objects(read, write) through any public api's or apps(like django apps). so, if you want to access s3 objects in the particular bucket you should set the permission to be publicly accessible(see the permission section of bucket). For further control you can add ACL(Access control list) users from the ACL section.

you can refer to this link

Answer 9

Setting AWS_DEFAULT_ACL = None worked for me. It looks like boto requests public-read ACL by default so unless you have made your bucket public it won't work.