CODEWITHSUNDEEP
amazon-web-servicesaws-cloudformationamazon-emr
Kristian
Kristian

Reputation: 21840

How to designate spot instance pricing in a CloudFormation script

I'm creating an EMR cluster within a CloudFormation script, and I am successfully able to run it to completion and get the stack to build, but now I'm wondering how to implement spot pricing in the CF template as well.

Here's what I'm working with for the cluster creation:

"Resources": {
    "MyCluster": {
        "Type": "AWS::EMR::Cluster",
        "Properties": {
            "Applications": [
                { "Name" : "Hadoop" },
                { "Name" : "SPARK" },
                { "Name" : "Ganglia" }
            ],
            "BootstrapActions" : [...],
            "Instances": {
                "AdditionalMasterSecurityGroups" : [{ "Fn::GetAtt" : ["rAllowJupyter","GroupId"] }],
                "Ec2KeyName" : { "Ref" : "EC2KeyName" },
                "Ec2SubnetId" : { "Ref" : "Subnet" },
                "MasterInstanceGroup": {
                    "InstanceCount": 1,
                    "InstanceType": { "Ref" : "InstanceType" }
                },
                "CoreInstanceGroup": {
                    "InstanceCount": { "Ref" : "CoreNodeCount" },
                    "InstanceType": { "Ref" : "InstanceType" }
                }
            },
            "Configurations": [...],
            "Name": "MyCluster",
            "JobFlowRole": "EMR_EC2_DefaultRole",
            "ServiceRole": "EMR_DefaultRole",
            "ReleaseLabel": "emr-4.6.0",
            "LogUri": "s3://path/to/logs/",
            "Tags": [
                { "Key": "Name", "Value": "aqa-spark"},
                { "Key": "Owner",   "Value": { "Ref" : "OwnerTag" }},
                { "Key": "Purpose", "Value": { "Ref" : "PurposeTag" }}
            ]
        }
    }
},

Are there parameters that I can use to designate spot-instances within an EMR cluster for my CloudFormation script?

Upvotes: 0

Views: 944

Answers (1)

Daniel van den Berg
Daniel van den Berg

Reputation: 56

You can make the MasterInstanceGroup (or any other) use spot instances by adding a Market and BidPrice parameter to the configuration.

For example, to have r3.large spot instances for $0.10:

"MasterInstanceGroup": {
  "InstanceCount": 1,
  "InstanceType": "r3.large",
  "Market": "SPOT",
  "BidPrice": "0.10"
}

Source: Amazon EMR Cluster JobFlowInstancesConfig InstanceGroupConfig documentation

Upvotes: 1

Related Questions