CODEWITHSUNDEEP
amazon-web-servicesamazon-ec2
Vame
Vame

Reputation: 2063

Launch multiple EC2 instances in parallel

I want to launch a number of EC2 machines in parallel. Until now I was using boto and fabric but the serial execution takes ages to launch and provision them one by one. Is there any alternative solution to do that?

Upvotes: 5

Views: 7625

Answers (2)

iGili
iGili

Reputation: 883

you can use CloudFormation to launch an auto scaling group with fixed size:

"MyFixedSizeGroup":{
        "Type":"AWS::AutoScaling::AutoScalingGroup",
        "Properties":{
            "LaunchConfigurationName":{"Ref":"GlobalWorkersSmallLaunchConf"},
            "AvailabilityZones" : [ "us-east-1a" ],
            "MinSize":"4",
            "MaxSize":"4",
            "DesiredCapacity":"4",
            "Tags":[{"Key":"Name", "Value":"worker instance", "PropagateAtLaunch":"true"}]          
        }           
}

and the desired launch configuration, for example:

"GlobalWorkersSmallLaunchConf":{
        "Type":"AWS::AutoScaling::LaunchConfiguration",
        "Properties":{"KeyName":{"Ref":"MyKeyName"},
                      "ImageId":"ami-SomeAmi",
                      "UserData":{"Fn::Base64":{"Fn::Join":["",[{"Ref":"SomeInitScript"}]]}},
                      "SecurityGroups":[{"Ref":"InstanceSecurityGroup"}],
                      "InstanceType":"m1.small",
                      "InstanceMonitoring":"false"
        }           
}

You can use it with boto or the via the CLI

BTW- It performs a lot better because you send a single request to AWS service, that is handled as a stack. to terminate the instances (and any other resources you want to add) just delete the stack.

Upvotes: 1

chris
chris

Reputation: 37490

The amazon command line tools support a parameter for number of instances.

aws ec2 run-instances help
   --count (string)
      Number of instances to launch. If a single number is provided, it is
      assumed  to  be the minimum to launch (defaults to 1). If a range is
      provided in the form min:max then the first number is interpreted as
      the  minimum  number of instances to launch and the second is inter-
      preted as the maximum number of instances to launch.

If you're running the older CLI:

ec2-run-instances
 -n, --instance-count MIN[-MAX]
      The number of instances to attempt to launch. May be specified as a
      single integer or as a range (min-max). This specifies the minimum
      and maximum number of instances to attempt to launch. If a single
      integer is specified min and max are both set to that value.

Update: According to the boto ec2 documentation, you can pass in min_count and max_count parameter to the run_instances command, which would also let you start multiple instances in parallel.

Upvotes: 3

Related Questions