CODEWITHSUNDEEP
amazon-web-servicesdockeramazon-ec2architectureamazon-elastic-beanstalk
mfnight
mfnight

Reputation: 118

Scaling # of docker containers in AWS elastic beanstalk per EC2 instance [app architecture]

I have a dockerized data application which runs long calculations and doesn't have much flexibility with multi-threading or async, thus multiple users is a growing problem. The docker image currently runs on Elastic Beanstalk (mainly just to familiarize myself with AWS) but I quickly noticed that horizontally scaling the EC2 instances, running at 1 docker container per EC2 instance, will be extremely costly. The app's calculations aren't extremely CPU-intensive, so it seems wise to instead take advantage of the number of containers per instance.

Does Elastic Beanstalk have a strategy for deploying 'x' # of docker containers per EC2 instance and directing traffic accordingly? Or should I be focusing on a different AWS product entirely? I'd like to have 10-20 copies of the same docker image available at a given time to handle traffic but not achieve this by directly increasing the # of EC2 instances to 10-20 with the load balancer settings.

As an aside, I know you can run multi-container apps by specifying such in your dockerrun.aws.json file, but my understanding is that you would do this primarily for deploying a single app that requires multiple different dockerized services. Are there instead config settings I can add to my dockerrun file to achieve what I'm looking for? ...without hard-coding 20 different copies of the image+port-mapping

Dockerrun.aws.json

{
"AWSEBDockerrunVersion": "1",
"Authentication": {
    "Bucket": "s3_bucket_name",
    "Key": "config_file"
},
"Image": {
    "Name": "image_name"
},
"Ports": [{
    "ContainerPort": 3838,
    "HostPort": 3838
}]

}

Upvotes: 0

Views: 706

Answers (1)

Adiii
Adiii

Reputation: 59966

Different horses for different courses, you can use Elastic Beanstalk for the normal workload but you can not expect that much scalability and flexibility that ECS provides in case of Docker.

Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service. Customers such as Duolingo, Samsung, GE, and Cookpad use ECS to run their most sensitive and mission critical applications because of its security, reliability, and scalability.

For you query

I'd like to have 10-20 copies of the same docker image available at a given time to handle traffic but not achieve this by directly increasing the # of EC2 instances to 10-20 with the load balancer settings.

This is extremely easy with ECS, all you need to tell the replica number of your task and ECS will manage that accordingly in one instance or across all instance in the cluster.

without hard-coding 20 different copies of the image+port-mapping

You do not need to create 20 copies, just one copy of task definition with dynamic port and place load balancer on top the service, ECS will manage the port so you do not need to worry about hardcoded and port conflict. enter image description here

Plus you can also go fargate which serverless container services and you will only manage application "Zero server to manage"

enter image description here

new-launch-introducing-aws-fargate

Upvotes: 3

Related Questions