AWS Instance Fleets with multiple ec2 types

Question

I am getting ready to move my team to AWS instance fleets. Thus far in testing, it's been great; it has really helped with capacity issues via instance diversification.

However, I expected AWS would choose a mix of instances based on the types I supply to the instance fleet template. For example, I target 15 core nodes as either i3.2xlarge or r5d.2xlarge in the template. So, I would expect my fleet to include some mix of i3.2xlarge and r5d.2xlarge instances. Instead, my fleet is exclusively supplied with r5d.2xlarge machines.

Perhaps this behavior is due to default allocation strategies, and that there is truly more supply of r5d.2xlarge than i3.2xlarge, which is why no i3.2xlarge machines were used.

But my question is: for a core node within an instance fleet, can that core node include different types of instances?

EDIT:

I am using python and boto3 via run_job_flow() to create the cluster. I am using the templates described here in boto3 docs.

When I run the below, all of my core instances are of the same instance type. I tried setting the AllocationStrategy to diversified, but received an error unknown parameter error. After reading the boto3 docs, it sounds like that isn't supported.

So how would I adjust this template to create different instance types in the same core node in the same cluster?

The core node dict I'm submitting in the InstanceFleets list looks like this (keep in mind some of the key values are variables):

'Name': 'Core Node',
'InstanceFleetType': 'CORE',
'InstanceTypeConfigs': [
    { 
        'EbsConfiguration': { 
            'EbsBlockDeviceConfigs': [ 
                { 
                    'VolumeSpecification': {
                        'SizeInGB': core_ebs_volume_gb_size,
                        'VolumeType': core_ebs_volume_type
                    },
                    'VolumesPerInstance': core_ebs_volumes
                }
            ],
            'EbsOptimized': True
        },
        'InstanceType': 'i3.2xlarge',
        'WeightedCapacity': 1
    },
    { 
        'EbsConfiguration': { 
            'EbsBlockDeviceConfigs': [ 
                { 
                    'VolumeSpecification': {
                        'SizeInGB': core_ebs_volume_gb_size,
                        'VolumeType': core_ebs_volume_type
                    },
                    'VolumesPerInstance': core_ebs_volumes
                }
            ],
            'EbsOptimized': True
        },
        'InstanceType': 'r5d.2xlarge',
        'WeightedCapacity': 1
    },
    { 
        'EbsConfiguration': { 
            'EbsBlockDeviceConfigs': [ 
                { 
                    'VolumeSpecification': {
                        'SizeInGB': core_ebs_volume_gb_size,
                        'VolumeType': core_ebs_volume_type
                    },
                    'VolumesPerInstance': core_ebs_volumes
                }
            ],
            'EbsOptimized': True
        },
        'InstanceType': 'c5d.2xlarge',
        'WeightedCapacity': 1
    },
],
'TargetOnDemandCapacity': 3,
'LaunchSpecifications': {
    'SpotSpecification': {
        'TimeoutDurationMinutes': 60,
        'TimeoutAction': 'SWITCH_TO_ON_DEMAND',
        'AllocationStrategy': 'capacity-optimized'
    }
}

Answer 1

You can mix instance types and set weight by setting EC2 Fleet instance weighting .

This docs page shows some examples for CLI settings, like this.

aws emr create-cluster --release-label emr-5.3.1 --service-role EMR_DefaultRole \
--ec2-attributes InstanceProfile=EMR_EC2_DefaultRole,SubnetIds=['subnet-ab12345c','subnet-de67890f'] \
--instance-fleets InstanceFleetType=MASTER,TargetOnDemandCapacity=1,InstanceTypeConfigs=['{InstanceType=m5.xlarge}'] \
InstanceFleetType=CORE,TargetSpotCapacity=11,InstanceTypeConfigs=['{InstanceType=m5.xlarge,BidPrice=0.5,WeightedCapacity=3}',\
'{InstanceType=m4.2xlarge,BidPrice=0.9,WeightedCapacity=5}'],\
LaunchSpecifications={SpotSpecification='{TimeoutDurationMinutes=120,TimeoutAction=SWITCH_TO_ON_DEMAND}'}

In this setting, WeightedCapacity=5 for m4.2xlarge and WeightedCapacity=3 for m5.xlarge.