CODEWITHSUNDEEP
amazon-web-servicesdockeramazon-ec2amazon-elastic-beanstalkamazon-ecs
Sean Pianka
Sean Pianka

Reputation: 2325

Scheduling the stopping/starting of an EC2 instance when not in use by a Beanstalk Deployment or an ECS task?

I have a Docker image containing Python code and third-party binary executables. There are only outbound network requests. The image must run hourly and each execution lasts ~3 minutes.

I can:

  1. Use an EC2 instance and schedule hourly execution via cron
  2. Create a CloudWatch Event/Rule to run an ECS Task Defintion hourly
  3. Setup an Elastic Beanstalk environment and schedule hourly deployment of the image

In all of these scenarios, an EC2 instance is running 24/7 and I am being charged for extended periods of no usage.

How do I accomplish scheduling the starting of an existing EC2 instance hourly and the stopping of said instance after the completion of my docker image?

Upvotes: 2

Views: 1482

Answers (1)

user189198
user189198

Reputation:

Here's one approach I can think of. It's very high-level, and omits some details, but conceptually it would work just fine. You'll also need to consider the Identity & Access Management (IAM) Roles used:

  • CloudWatch Event Rule to trigger the Step Function
  • AWS Step Function to trigger the Lambda function
  • AWS Lambda function to start up EC2 instances

  • EC2 instance polling the Step Functions service for Activity Tasks

    1. Create a CloudWatch Event Rule to schedule a periodic task, using a cron expression
    2. The Target of the CloudWatch Event Rule is an AWS Step Function
    3. The AWS Step Function State Machine starts by triggering an AWS Lambda function, which starts the EC2 instance
    4. The next step in the Step Functions State Machine invokes an Activity Task, representing the Docker container that needs to execute
    5. The EC2 instance has a script running on it, which polls the Activity Task for work
    6. The EC2 instance executes the Docker container, waits for it to finish, and sends a completion message to the Step Functions Activity Task
    7. The script running on the EC2 instance shuts itself down
    8. The AWS Step Function ends

Keep in mind that a potentially better option would be to spin up a new EC2 instance every hour, instead of simply starting and stopping the same instance. Although you might get better startup performance by starting an existing instance vs. launching a new instance, you'll also have to spend time to maintain the EC2 instance like a pet: fix issues if they crop up, or patch the operating system periodically. In today's world, it's a commonly accepted practice that infrastructure should be disposable. After all, you've already packaged up your application into a Docker container, so you most likely don't have overly specific expectations around which host that container is actually being executed on.

Another option would be to use AWS Fargate, which is designed to run Docker containers, without worrying about spinning up and managing container infrastructure.

Upvotes: 1

Related Questions