CODEWITHSUNDEEP
amazon-web-servicestestingamazon-ec2continuous-integrationgpu
tamuhey
tamuhey

Reputation: 3535

How to automatically start, execute and stop EC2?

I want to test my Python library in GPU machine once a day. I decided to use AWS EC2 for testing. However, the fee of gpu machine is very high, so I want to stop the instance after the test ends.

Thus, I want to do the followings once a day automatically

  1. Start EC2 instance (which is setup manually)
  2. Execute command (test -> push logs to S3)
  3. Stop EC2 (not remove)

How to do this?

Upvotes: 4

Views: 3783

Answers (4)

jotaelesalinas
jotaelesalinas

Reputation: 1497

I created a shell script to start an EC2 instance -if not already running,- connect via SSH and, if you want, run a command.

https://gist.github.com/jotaelesalinas/396812f821785f76e5e36cf928777a12

You can use it in three different ways:

./ec2-start-and-ssh.sh -i <instance id> -s

will show status information about your instance: running state and private and public IP addresses.

./ec2-start-and-ssh.sh -i <instance id>

will connect and leave you inside the default shell.

./ec2-start-and-ssh.sh -i <instance id> <command>

will run whatever command you specify, e.g.:

./ec2-start-and-ssh.sh -i <instance id> ./run.sh

./ec2-start-and-ssh.sh -i <instance id> sudo poweroff

I use the last two commands to run periodic jobs minimizing billing costs.

I hope this helps!

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 269370

It is very simple...

Run script on startup

To run a script automatically when the instance starts (every time it starts, not just the first time), put your script in this directory:

/var/lib/cloud/scripts/per-boot/

Stop instance when test has finished

Simply issue a shutdown command to the operating system at the end of your script:

sudo shutdown now -h

Upvotes: 5

matesio
matesio

Reputation: 1604

You can push script logs to custom coudwatch namespaces. Like when the process ends publish a state to cloudwatch. In cloudwatch create alarms based on the state of process, so if it has a completed state trigger an AWS lambda function that will stop instance after completion of your job.

Also if you want to start and stop on specific time you can use ec2 instance scheduler to start/stop instances. It just works like a cron job at specific intervals.

Upvotes: 1

Ghonima
Ghonima

Reputation: 3082

You can use the aws cli

To start an instance you would do the following

aws ec2 start-instances --instance-ids i-1234567890abcdef0

and to stop the instance you would do the following

aws ec2 stop-instances --instance-ids i-1234567890abcdef0

To execute commands inside the machine, you will need to ssh into it and run the commands that you need, then you can use the aws cli to upload files to s3

aws s3 cp test.txt s3://mybucket/test2.txt

I suggest reading the aws cli documentation, you will find most if not all what you need to automate aws commands there.

Upvotes: 0

Related Questions