CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-cloudformationaws-cloudformation-custom-resource
olive
olive

Reputation: 211

How to upload contents into S3 bucket via CloudFormation template using custom resource?

Fairly new to cloudformation templating but all I am looking to create a template where I create a S3 bucket and import contents into that bucket from another S3 bucket from a different account (that is also mine). I realize CloudFormation does not natively supports importing contents into S3 bucket, and I have to utilize custom resource. I could not find any reference/resources that does such tasks. Hoping someone could point out some examples or maybe even some guidance as to how to tackle this.

Thank you very much!

Upvotes: 0

Views: 2248

Answers (2)

John Rotenstein
John Rotenstein

Reputation: 269340

If Custom Resources scare you, then a simpler way is to launch an Amazon EC2 instance with a startup script specified via User Data.

The CloudFormation template can 'insert' the name of the new bucket into the script by referencing the bucket resource that was created. The script could then run an AWS CLI command to copy the files across.

Plus, it's not expensive. A t3.micro instance is about 1c/hour and it is charged per second, so it's pretty darn close to free.

Upvotes: 0

Marcin
Marcin

Reputation: 238209

Can't provide full code, but can provide some guidance. There are few ways of doing this, but I will list one:

  1. Create a bucket policy for the bucket in the second account. The policy should allow the first account (one with cfn) to read it. There are many resources on doing this. One from AWS is here.

  2. Create a standalone lambda function in the first account with execution role allowing it to the read bucket from the second account. This is not a custom resource yet. The purpose of this lambda function is to test the cross-account permissions, and your code which reads objects from it. This is like a test function to sort out all the permissions and polish object copying code from one bucket to other.

  3. Once your lambda function works as intended, you modify it (or create new one) as a custom resource in CFN. As a custom resource, it will need to take your newly created bucket in cfn as one of its arguments. For easier creation of custom resources this aws helper can be used.

Note, that the lambda execution timeout is 15 minutes. Depending on how many objects you have, it may be not enough.

Hope this helps.

Upvotes: 3

Related Questions