What is better: configure instance on launch or launch a pre-backed image?

Question

I am developing a cloud solution. I have no experience in it, so I want to ask some professionals about best practices. The current question mostly related to the autoscaling groups functionality.

I've read a lot of howtos and guides and came to conclusion that the only ways to provision/configure instances in ASG are:

• to pre-bake AMI;
• to use user_data field.

So, let's assume, I have an autoscaling group. And I want to configure instances what it launches, for example, using chef-solo (or ansible-local, but as I understood, chef is better option for the aws).

I see only 2 ways how to do this:

• Use packer and pre-bake image locally (using chef-solo provider), then update ASG configuration with the brand-new created AMI;
• Use base Amazon AMI and configure images at the launch, using user_data script: install chef-solo, fetch cookbooks from the git, run chef-solo on machine.

What is better choice in you opinion and why? Also I am interested in how to update already running instances in the ASG when my chef cookbooks configuration changes.

Also, if you know better options, leave them here. I am open to discuss.

Answer 1

It depends on your use case.

A pre-baked AMI may be quicker to launch when scaling up, but if you need to make even small changes to the code or configuration, you'll need to bake another AMI. Using user data (whether using straight OS commands or Chef or something else) may take longer if you're installing application servers and deploying applications, and you may also be introducing external dependencies for scaling: what if the GitHub repository is off line or a necessary download is blocked?

So, if speed of scale-up is important, consider a pre-baked AMI. If you can tolerate a reasonable scale-up hit, look at a hybrid approach:

• Bake into your AMI the Chef DK and any other large objects you need. For example, you might bake your application server installation into the AMI and then just have Chef configure it through user data.
• Make sure your dependencies, scripts and deployables such as WAR files are in reliable repositories such as S3.

The best advice is to try both approaches to get some metrics and see how these fit your use cases.