Grant EC2 instance access to S3 Bucket

Question

I want to grant my ec2 instance access to an s3 bucket.

On this ec2 instance, a container with my application is launched. Now I don't get permission on the s3 bucket.

This is my bucket policy

{
"Version": "2012-10-17",
"Id": "Policy1462808223348",
"Statement": [
    {
        "Sid": "Stmt1462808220978",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::714656454815:role/ecsInstanceRole"
        },
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::bucket-name/*",
        "Condition": {
            "IpAddress": {
                "aws:SourceIp": "private-ip/32"
            }
        }
    }
]
}

But it doesn't work until I give the bucket the permission for everyone to access it.

I try to curl the file in the s3 bucket from inside the ec2 instance but this doesn't work either.

Answer 1

I faced the same problem. I finally resolved it by creating an access-point for the bucket in question using AWS CLI see https://docs.aws.amazon.com/AmazonS3/latest/dev/creating-access-points.html and I then created a bucket policy like following

{
    "Version": "2012-10-17",
    "Id": "Policy1583357393961",
    "Statement": [
        {
            "Sid": "Stmt1583357315674",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account-id>:role/ecsInstanceRole"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<your-bucket>"
        },
        {
            "Sid": "Stmt1583357391961",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account-id>:role/ecsInstanceRole"
            },
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::<your-bucket>/*"
        }
    ]
}

Please make sure you are using a newer version of aws cli (1.11.xxx didn't work for me). I finally installed the version 2 of cli to get this to work.

Answer 2

at least of now, 2019, there is a much easier and cleaner way to do it (the credentials never have to be stored in the instance, instead it can query them automatically):

1. create an IAM Role for your instance and assign it
2. create a policy to grant access to your s3 bucket
3. assign the policy to the instance's IAM role
4. upload/download objects e.g. via aws cli for s3 - cp e.g. aws s3 cp <S3Uri> <LocalPath>

@2: An example of a JSON Policy to Allow Read and Write Access to Objects in an S3 Bucket is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListObjectsInBucket",
            "Effect": "Allow",
            "Action": ["s3:ListBucket"],
            "Resource": ["arn:aws:s3:::bucket-name"]
        },
        {
            "Sid": "AllObjectActions",
            "Effect": "Allow",
            "Action": "s3:*Object",
            "Resource": ["arn:aws:s3:::bucket-name/*"]
        }
    ]
}

You have to adjust allowed actions, and replace "bucket-name"

Answer 3

There is no direct way of granting "EC2" instance access to AWS server, but you can try the following.

1. Create a new user in AWS IAM, and download the credentials file.
2. This user will represent your EC2 server.
3. Provide the user with permissions to your S3 Bucket.

4. Next, place the credentials file in the following location:-
   EC2 - Windows Instance:
   a. Place the credentials file anywhere you wish. (e.g. C:/credentials)
   b. Create an environment variable AWS_CREDENTIAL_PROFILES_FILE and put the value as the path where you put your credentials file (e.g. C:/credentials)
   EC2 - Linux Instance
   a. Follow steps from windows instance
   b. Create a folder .aws inside your app-server's root folder (e.g. /usr/share/tomcat6).
   c. Create a symmlink between your environment variable and your .aws folder sudo ln -s $AWS_CREDENTIAL_PROFILES_FILE /usr/share/tomcat6/.aws/credentials

5. Now that your credentials file is placed, you can use Java code to access the bucket.
   NOTE: AWS-SDK libraries are required for this

    AWSCredentials credentials = null;
            try {
                credentials = new ProfileCredentialsProvider().getCredentials();
            } catch (Exception e) {
                LOG.error("Unable to load credentials " + e);
                failureMsg = "Cannot connect to file server.";
                throw new AmazonClientException(
                        "Cannot load the credentials from the credential profiles file. " +
                        "Please make sure that your credentials file is at the correct " +
                        "location (environment variable : AWS_CREDENTIAL_PROFILES_FILE), and is in valid format.",
                        e);
            }

            AmazonS3 s3 = new AmazonS3Client(credentials);
            Region usWest2 = Region.getRegion(Regions.US_WEST_2);
            s3.setRegion(usWest2);
    ObjectListing objectListing = s3.listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix(prefix));

Where bucketName = [Your Bucket Name]
and prefix = [your folder structure inside your bucket, where your file(s) are contained]

Hope that helps. Also, if you are not using Java, you can check out AWS-SDKs in other programming languages too.

Answer 4

I found it out....

It only works with the public IP from the ec2 instance.

Answer 5

Try this:

{
"Version": "2012-10-17",
"Id": "Policy1462808223348",
"Statement": [
    {
        "Sid": "Stmt1462808220978",
        "Effect": "Allow",
        "Principal": {
            "AWS": "*" 
        },
        "Action": "s3:*",
        "Resource": "arn:aws:s3:::bucket-name/*",
        "Condition": {
            "IpAddress": {
                "aws:SourceIp": "yourIp/24"
            }
        }
    }
]
}