CODEWITHSUNDEEP
amazon-s3amazon-athena
Dean
Dean

Reputation: 11

Permission denied on S3 path - What are the minimum policies required on the data source to get athena-express to work?

I'm attempting to use the Athena-express node module to query Athena.

Per the Athena-express docs:

This IAM role/user must have AmazonAthenaFullAccess and AmazonS3FullAccess policies attached Note: As an alternative to granting AmazonS3FullAccess you could granularize and limit write access to a specific bucket. Just specify this bucket name during athena-express initialization"

Providing AmazonS3FullAccess to this micro service is a non-starter. What is the minimum set of priviledges I can grant to the micro service and still get around the "Permission denied on S3 path: s3://..." errors I've been getting?

Currently, I've got the following

Output location: (I don't think the problem is here)

s3:AbortMultipartUpload, s3:CreateMultipartUpload, s3:DeleteObject, s3:Get*, s3:List*, s3:PutObject, s3:PutObjectTagging on "arn:aws:s3:::[my-bucket-name]/tmp/athena" and "arn:aws:s3:::[my-bucket-name]/tmp/athena/*"

Data source location:

s3:GetBucketLocation on "arn:aws:s3:::*"

s3:ListBucket on "arn:aws:s3:::[my-bucket-name]"

s3:Get* and s3:List* on "arn:aws:s3:::[my-bucket-name]/production/[path]/[path]" and "arn:aws:s3:::[my-bucket-name]/production/[path]/[path]/*"

The error message I get with the above is: "Permission denied on S3 path: s3://[my-bucket-name]/production/[path]/[path]/v1/dt=2022-05-26/.hoodie_partition_metadata"

Any suggestions? Thanks!

Upvotes: 0

Views: 655

Answers (1)

Dean
Dean

Reputation: 11

It turned out that the bucket storing the data I needed to query was encrypted, which meant that the missing permission to query was kms:Decrypt.

Athena by outputs the results of a query to a location (which athena-express then retrieves). The location of the output was in that same encrypted bucket, so I also ended up giving my cronjob kms:Encrypt and kms:GeneratedDataKey.

I ended up using CloudTrails to figure out which permissions were causing my queries to fail.

Upvotes: 1

Related Questions