CODEWITHSUNDEEP
amazon-ec2amazon-web-services
Tihom
Tihom

Reputation: 3394

Running a script on an AWS server

I have a script that I need to run once a day that requires a lot of memory. I would like to run it on a dedicated amazon box.

Is there some automated way to build a box, download all required software (like ruby) and then run my script. After the script is ran, I would like to shutdown the box.

The two options I can think of are:

  1. I am thinking about hacking EMR to do this. (My script is a mapper against an empty directory)
  2. Chef - This seemed like too much for one simple script.

Upvotes: 6

Views: 5662

Answers (1)

Eric Hammond
Eric Hammond

Reputation: 22407

You can accomplish setting up a new EC2 instance on startup using the official Ubuntu AMIs, the official Amazon Linux AMIs, and any other AMI that supports the concept of a user-data script.

Create a script (bash, Perl, Python,

  1. whatever) that starts with #!
  2. Pass this script as the user-data when running the EC2 instance.
  3. The script will automatically be run as root on the first boot.

Here's the article where I introduced the concept of a user-data script:

Automate EC2 Instance Setup with user-data Scripts
http://alestic.com/2009/06/ec2-user-data-scripts

Your user-data script can install the required software, configure it, install your work script, and set up a cron job that runs the work script once a day.

ENHANCEMENT:

If the installation script don't take a long time to run (e.g., under an hour or few) then you don't even have to run a single dedicated instance 24 hours a day. You can instead use an approach that lets AWS start an instance for you on a regular schedule.

Here's an article I wrote that provides details on this approach with sample commands:

Running EC2 Instances on a Recurring Schedule with Auto Scaling
http://alestic.com/2011/11/ec2-schedule-instance

The general approach is to use Auto Scaling to start an instance with your user-data script on a regular schedule. Your job will terminate the instance when it has completed. They key is to suspend Auto Scaling's normal desire to re-start instances that terminate so that you don't pay for a running instance until the next time your job starts.

Upvotes: 9

Related Questions