ELI5: How to grant a single user full access to an S3 bucket?

Question

I've read a ton of AWS documentation, and tried dozens of sample configurations, but the only AWS policy I've managed to get working is the one that grants everyone full-access to a S3 bucket. This is terrifying.

The single-user full access scenario:

Here's the setup:

1. An AWS user, jenkins, who doesn't belong to any groups or roles.
2. A server containing jenkins AWS_SECRET_KEY and AWS_ACCESS_KEY.
3. An brand new empty S3 bucket, notes using default permissions.

The question is whether it is possible to use curl to implement the following operations using the credentials associated with the jenkins user:

1. Upload a text file, foo containing the phrase "hello, world" to notes.
2. Download the text file, foo, from notes.
3. Fetch a list of all of the files in notes.

What a good solution looks like:

An acceptable solution to my question has the following parts:

1. A set of curl commands demonstrating the solution.
2. An explanation of any S3 settings (i.e. ACLs) or IAM user/group/role policies and how these damn things are supposed to work.
3. [Optional] References to 3rd-party blogs/posts from 2014 onwards containing sound explanations of how these security parts fit together.

However, solutions like "use SDK X" or "download tool Y" are nice, but ultimately unhelpful, and solutions like "what are you really trying to do here, maybe there is an easier way" will be met with a comment referring to this section. Thank you for understanding that I really just want 3 curl statements and an explanation of S3 ACLs and IAM policies for this scenario.

Answer 1

I was looking for a place to share this solution to my own headaches, putting it here as it partly matches your question.

I think the aws policy generators are novice-unfriendly. I was getting a lot of access denied and not understanding why. Unconstructive message IMO as it doesn't say what policy you violate. It turned out the lib was also setting public read rights for files uploaded, a.k.a ACL. Long story short, I was missing PutObjectACL. Here are some examples for setting up a single S3 bucket access with AWS IAM, without giving access to any other bucket.

SERVER Policy

ie IAM user myapp-staging { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:ListMultipartUploadParts", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::myappbucket-staging" ] }, { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:PutObjectAcl" ], "Resource": [ "arn:aws:s3:::myappbucket-staging/*" ] } ] }

---

Human Policy

Here's a policy for an human IAM user to use AWS S3 console or other S3 viewer, to verify the content uploaded, without exposing contents of other buckets (it'll list bucket names though, can't figure out how to limit that and I don't really care about it).

IAM user myuser with Policy MyappTeamPolicy:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets" ], "Resource": [ "arn:aws:s3:::*" ] }, { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:ListMultipartUploadParts", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::myappbucket", "arn:aws:s3:::myappbucket-staging" ] }, { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:GetObjectAcl", "s3:ListMultipartUploadParts", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::myappbucket/*", "arn:aws:s3:::myappbucket-staging/*" ] } ] }

Answer 2

There are 2 parts to getting this to work: building the request and setting the correct policy on the S3 bucket.

The easy part: For building the bucket policy you can use http://awspolicygen.s3.amazonaws.com/policygen.html
You can see examples here: http://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html

For building the request. There is a lot that goes into building the request with the most challenging part being signing the request. You won't be able to use curl only, but you can get away without using an sdk. Here is an example of all needs to happen for you to be able to issue the final curl: http://geek.co.il/2014/11/19/script-day-amazon-aws-signature-version-4

In the end you see the final curl. Don't forget to update for s3 (example is for SQS, but that's basically all the grunt work that needs to be done if you don't want or cannot use an sdk).