CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-athena
Alex
Alex

Reputation: 977

Access denied to S3 bucket from Athena although permissions = allow all

I followed the tutorial http://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html. I set up a IAM user (with strange permissions) and a S3-bucket and could query sample Athena tables and the outputs were written to my S3-bucket. Now I have credentials from a client to access his Athena table. This doesn't work and I get following error:

Access denied when writing output to url: s3://my-test-bucket/b36-f3c0-482-a225-34d63d355.txt . Please ensure you are allowed to access the S3 bucket. If you are encrypting query results with KMS key, please ensure you are allowed to access your KMS key

My S3-bucket is as public as it gets. Permissions for "Any authenticated AWS user" : Read, Write. Permissions for "everyone" : Read, Write. Permissions for "Log delivery" : Read, Write. "Permission Access" for everything: Read, Write

Bucket Policy allows everyone to do everything.

{
 "Version": "2012-10-17",
 "Statement": [
     {
         "Effect": "Allow",
         "Principal": "*",
         "Action": "s3:ListBucket",
         "Resource": "arn:aws:s3:::my-test-bucket"
     },
     {
         "Effect": "Allow",
         "Principal": "*",
         "Action": [
             "s3:PutObject",
             "s3:GetObject",
             "s3:DeleteObject"
         ],
         "Resource": "arn:aws:s3:::my-test-bucket/*"
     },
     {
         "Sid": "AddPerm",
         "Effect": "Allow",
         "Principal": "*",
         "Action": [
             "s3:GetObject",
             "s3:PutObject"
         ],
         "Resource": "arn:aws:s3:::my-test-bucket/*"
     }
 ] }

CORS configuration:

 <?xml version="1.0" encoding="UTF-8"?> <CORSConfiguration
 xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <CORSRule>
     <AllowedOrigin>*</AllowedOrigin>
     <AllowedMethod>PUT</AllowedMethod>
     <AllowedMethod>POST</AllowedMethod>
     <AllowedMethod>DELETE</AllowedMethod>
     <AllowedHeader>*</AllowedHeader> </CORSRule> </CORSConfiguration>

Still, I get the error: Access denied when writing output to url...

Edit: From time to time I get an other error instead: "Unable to verify/create output bucket my-test-bucket". Not sure why I get different errors.

What can I do?

Upvotes: 7

Views: 10431

Answers (1)

wcyn
wcyn

Reputation: 4226

Had an almost similar issue with a particular file on s3. I could not read the file. Solved the issue by changing the access permissions of the file using the mv command and the --acl argument. Trying to access a file called data.jsonlines gave the ACCESS DENIED error. Solved it by running the following commands:

NOTE: You will need to have the AWS CLI installed: $ pip install --upgrade --user awscli

aws s3 cp s3://<s3 bucket name>/path/to/file/data.jsonlines  s3://cfa-opengazettes-ke/gazettes/data_copy.jsonlines 

aws s3 mv --acl public-read s3://<s3 bucket name>/path/to/file/data_copy.jsonlines s3://cfa-opengazettes-ke/gazettes/data.jsonlines

Or you can combine them by running:

aws s3 cp s3://<s3 bucket name>/path/to/file/data_out.jsonlines  s3://cfa-opengazettes-ke/gazettes/data_out2.jsonlines && aws s3 mv --acl public-read s3://cfa-opengazettes-ke/gazettes/data_out2.jsonlines s3://<s3 bucket name>/path/to/file/data_out.jsonlines

These commands carry out the steps below:

  • copy: s3://<s3 bucket name>/path/to/file/data.jsonlines to s3://c<s3 bucket name>/path/to/file/data_copy.jsonlines

  • move: s3://<s3 bucket name>/path/to/file/data_copy.jsonlines to s3://cfa-opengazettes-ke/path/to/file/data.jsonlines

Basically, it creates a copy of the file and then deletes it during the move while changing the permissions of the file.

Note the --acl option and the argument public-read. Maybe one of the below permissions could work for you. You can replace public-read with another permission. From the documentation:

--acl (string) Sets the ACL for the object when the command is performed. If you use this parameter you must have the "s3:PutObjectAcl" permission included in the list of actions for your IAM policy. Only accepts values of private, public-read, public-read-write, authenticated-read, aws-exec-read, bucket-owner-read, bucket-owner-full-control and log-delivery-write.

Some more useful information at this AWS page

Upvotes: 0

Related Questions