CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-ec2
David Salb
David Salb

Reputation: 162

How can I enable an ec2 instance to have private access to an S3 bucket?

First of all i'm aware of these questions:

but the solutions are not working for me.

I created a role "sample_role", attached the AmazonS3FullAccess-policy to it and assigned the role to the ec2-instance.

My bucket-policy is as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::My-Account-ID:role/sample_role"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::my_bucket/*"
        }
    ]
}

On my ec2-instance, listing my buckets works fine, both from the command line (aws s3 ls) and from python script. But when I try to upload a file test.txt to my bucket, I get AccessDenied:

import boto3

s3_client = boto3.client('s3')
s3_resource = boto3.resource('s3')
bucket = s3_resource.Bucket('my_bucket')

with open('test.txt', "rb") as f:
    s3_client.upload_fileobj(f, bucket.name, 'text.txt')

Error message:

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

Same happens when i just try to list the objects in my bucket. Command line aws s3api list-objects --my_bucket or python script:

import boto3

s3_resource = boto3.resource('s3')
bucket = s3_resource.Bucket('my_bucket')

for my_bucket_object in bucket.objects.all():
    print(my_bucket_object)

Error message:

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

When I turn off "Block all public access" in my bucket settings and enable public access in my access control list, it obviously works. But I need to restrict access to the specified role.

What am I missing? Thanks for your help!

Upvotes: 0

Views: 1036

Answers (3)

David Salb
David Salb

Reputation: 162

So I found the problem. The credentails of my ec2 instance were configured with the access key of a dev-user account to which the role was not assigned.

I found out by running aws sts get-caller-identity which returns the identity (e.g. IAM role) actually being used.

So it seems that the assigned role can be overwritten by the user identity, which makes sense. To solve the problem, I simply undid the configuration by deleting the configuration file ~/.aws/credentials. After that the identity changed to the assigned role.

Upvotes: 1

nickolay.laptev
nickolay.laptev

Reputation: 2565

When I turn off "Block all public access" in my bucket settings and enable public access in my access control list, it obviously works.

Remove "enable public access" from this sentence and this will be your solution :-)

"Block all public access" blocks all public access and it doesn't matter what bucket policy you use. So uncheck this option and your bucket policy will start working as you planned.

Upvotes: 1

John Rotenstein
John Rotenstein

Reputation: 269490

It appears that your requirement is:

  • You have an Amazon S3 bucket (my_bucket)
  • You have an Amazon EC2 instance with an IAM Role attached
  • You want to allow applications running on that EC2 instance to access my_bucket
  • You do not want the bucket to be publicly accessible

I will also assume that you are not trying to deny other users access to the bucket if they have already been granted that access. You are purely wanting to Allow access to the EC2 instance, without needing to Deny access to other users/roles that might also have access.

You can do this by adding a policy to the IAM Role that is attached to the EC2 instance:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::my_bucket",
                "arn:aws:s3:::my_bucket/*"
            ]
        }
    ]
}

This grants ALL Amazon S3 permissions to the IAM Role for my_bucket. Note that some commands require permission on the bucket itself, while other commands require permission on the contents (/*) of the bucket.

I should also mention that granting s3:* is probably too generous, because it would allow the applications running on the instance to delete content and even delete the bucket, which is probably not what you wish to grant. If possible, limit the actions to only those that are necessary.

Upvotes: 1

Related Questions