Managing Amazon EC2 instances

Question

I have started to use Amazon EC2 extensively and inorder to contain costs, I manually shutdown the instances before I leave work and bring them up when I come in. Sometimes I forget to shut them down. Is there a mechanism within the Amazon dashboard (or any other way) to automatically shut down the instances at say 6pm and bring them up at 6am? I am happy to write scripts or programs if there are any API's available. If you have some code written already, it would be great if you can share.

Answer 1

Using AWS Lambda is definitely simple with scheduled event. You can find the complete code script here

For example, to start EC2 instances

1. From AWS Console, configure a lambda function
2. Copy following code to "Edit code inline"

import boto3

def lambda_handler(event, context):
    client = boto3.client('ec2')
    response = client.start_instances(
        InstanceIds=[
            'i-xxxxxx',
            'i-xxxxxx',
        ]
    )
    print response

3. Give it basic execution role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "ec2:StartInstances"
      ],
      "Resource": [
        "arn:aws:logs:*:*:*",
        "arn:aws:ec2:*:*:*"
      ]
    }
  ]
}

4. Click create lambda function and add Event Source (CloudWatch events - schedule)

5. Finally, test your function to see if it work correctly

Answer 2

Can you just start with a simple solution like a cron job and AWS CLI?

start.sh

# some preparations
# ...
$(aws ec2 start-instances --instance-ids $LIST_OF_IDS)
$(aws ec2 wait instance-running --instance-ids $LIST_OF_IDS)

stop.sh

# some preparations
# ...
$(aws ec2 stop-instances --instance-ids $LIST_OF_IDS)
$(aws ec2 wait instance-stopped --instance-ids $LIST_OF_IDS)

Then crontab -e and add something like

* 6 * * 1,2,3,4,5 start.sh
* 18 * * 1,2,3,4,5 stop.sh

Answer 3

You don't need to write any scripts to achieve this, just use AWS console API. This is a perfect use case for AutoScaling with Scheduled Scaling. Steps:

1. Create a new Autoscaling group create a new autoscaling group. Example group: MY_AUTOSCALING_GROUP1
2. In your case you need to create 2 new recurring schedules create new schedule. Example:

Scale up every morning

aws autoscaling put-scheduled-update-group-action --scheduled-action-name scaleup-schedule --auto-scaling-group-name MY_AUTOSCALING_GROUP1 --recurrence "0 6 * * *" --desired-capacity 5

Scale down every evening

aws autoscaling put-scheduled-update-group-action --scheduled-action-name scaledown-schedule --auto-scaling-group-name MY_AUTOSCALING_GROUP1 --recurrence "0 18 * * *" --desired-capacity 0

5 EC2 instances will start every morning (6 AM) and terminate in evening (6 PM) everyday by applying above scheduled actions to your autoscaling group.

Answer 4

If your use case permits terminating and re-launching, you might consider Scheduled Scaling in an Auto Scaling group. AWS recently added tools to manage scheduling rules in the Management Console UI.

But I don't believe Auto Scaling will cover starting and stopping the same instance, preserving state.

Answer 5

I actually happened to have a cron job running for this. I'll put up my 'start instance' code here (for reference to get you started)

#!/usr/bin/python

import boto.ec2
import boto
import boto.regioninfo
import time
import sys
import boto.ec2.connection
from boto.ec2 import EC2Connection
from boto.regioninfo import RegionInfo
from boto import ec2
from boto import regioninfo

ec2_region = ec2.get_region(aws_access_key_id='QWERTACCESSKEY', aws_secret_access_key='QWERTSECRETKEY', region_name='ap-southeast-2')
conn = boto.ec2.connection.EC2Connection(aws_access_key_id='QWERTACCESSKEY', aws_secret_access_key='QWERTSECRETKEY', region=ec2_region)

instance1 = conn.get_only_instances(instance_ids=['i-xxxxxxx1'])
instance2 = conn.get_only_instances(instance_ids=['i-xxxxxxx2'])
instance3 = conn.get_only_instances(instance_ids=['i-xxxxxxx3'])
instance4 = conn.get_only_instances(instance_ids=['i-xxxxxxx4'])

def inst():  # updates states of instances
        instance1[0].update()
        instance2[0].update()
        instance3[0].update()
        instance4[0].update()

def startservers():
        inst()
        if instance1[0].state in 'stopped':
            instance1[0].start()
        if instance2[0].state in 'stopped':
            instance2[0].start()
        if instance3[0].state in 'stopped':
            instance3[0].start()
        if instance4[0].state in 'stopped':
            instance4[0].start()

def check(): # a double check to make sure the servers are down
    inst()
    while instance1[0].state in 'stopped' or instance2[0].state in 'stopped' or instance3[0].state in 'stopped' or instance4[0].state in 'stopped':
        startservers()
        time.sleep(30)

startservers()
time.sleep(60)
check()

This is my cronjob

31 8 * * 1-5 python /home/user/startaws

This runs at 8:31am from monday to friday.

Please Note This script works fine for me BUT it is definitely possible to make it much cleaner and simpler. There are also much better ways than this too. (I wrote this in a hurry so I'm sure I have some unnecessary lines of code in there) I hope it gives you an idea on how to start off :)

Answer 6

There are 2 solutions.

AWS Data Pipeline - You can schedule the instance start/stop just like cron. It will cost you one hour of t1.micro instance for every start/stop

AWS Lambda - Define a lambda function that gets triggered at a pre defined time. Your lambda function can start/stop instances. Your cost will be very minimal or $0

I used Data Pipeline for a long time before moving to Lambda. Data Pipeline is very trivial. Just paste the AWS CLI commands to stop and start instances. Lambda is more involved.