CODEWITHSUNDEEP
amazon-web-servicesboto
mike
mike

Reputation: 549

How do I limit access to S3 keys in a specific prefix? (In Python/boto)

I want to limit a boto program to being able to list keys and get keys only from a specific folder (prefix) in an S3 bucket.

I have an IAM role with the following access policy (with real names, of course). This is a variation on the example policy in the IAM docs for how to allow users to access a "home directory":

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": ["s3:Get*", "s3:List*"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::my-bucket"],
      "Condition":{"StringLike":{"s3:prefix":["my-prefix/*"]}}
    },
    {
       "Action":["s3:Get*", "s3:List*"],
       "Effect":"Allow",
       "Resource": ["arn:aws:s3:::my-bucket/my-prefix/*"]
    }
  ]
}

In boto, I connect to S3 and get temp creds. Then I run this:

bucket = s3_connection.get_bucket('my-bucket')
all_keys = bucket.get_all_keys()
for key in all_keys:
    print key

This fails on the get_bucket call. The only way I can get it to work so far is to remove the s3:Prefix condition (basically, remove the first statement in the policy). But that then allows the script to list keys from the entire bucket, not just those with a specific prefix. Am I using the wrong approach to getting the keys? All S3 access seems to begin with get_bucket.

I can make this work using the AWS CLI, like this:

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/my-role --role-session-name "role1" --profile other-account-admin-user > temp-creds.txt
# ... copy temp-creds into credentials file using profile name "temp-creds", then ...
# works
aws s3api list-objects --bucket my-bucket --prefix my-prefix/ --profile temp-creds 
# fails, as expected
aws s3api list-objects --bucket my-bucket --profile temp-creds

This leads me to think that there's a way to do this in boto, and I just can't figure it out. (Of course, there might be flaws in my testing, which happens rather a lot, haha.)

Upvotes: 4

Views: 1268

Answers (1)

garnaat
garnaat

Reputation: 45856

By default, the get_bucket method in boto attempts to verify that the bucket exists by performing a HEAD request on the bucket URL. I think this will fail in your case since the user doesn't have access to the root of the bucket.

So, I think something like this will work for you:

bucket = s3_connection.get_bucket('my-bucket', validate=False)
for key in bucket.list(prefix='my-prefix'):
    print(key)

Upvotes: 5

Related Questions