CODEWITHSUNDEEP
azureazure-hdinsight
Victor F
Victor F

Reputation: 915

Specifying HDInsight 3.2 Instance Size via powershell

According to new HDInsight pricing schema and release notes, you can now specify the size of the instance (A3-A9 and D3-D14). I, however, couldn't find a way of how to specify the instance size when creating a new cluster via powershell New-AzureHDInsightCluster command. I did find that see that now it accepts -DataNodeVMSize, however it doesn't understand keywords like "A7", and when specifying "Extra Large" it creates a standard A3 cluster. Any suggestions whether it's actually possible to specify instance size at the moment?

The "available" VM sizes https://msdn.microsoft.com/en-us/library/azure/dn197896.aspx

Here is the script I am running:

$VmSize = "Small";

New-AzureHDInsightCluster -Name $clusterName `
              -ClusterType Hadoop `
              -Version $MyClusterVersion `
              -Location $MyClusterLocation `
              -ClusterSizeInNodes $NumClusterNodes `
              -Credential $HdInsightCreds `
              -DefaultStorageAccountName $DefaultStorageAccountFqdn `
              -DefaultStorageAccountKey $storageAccountKey `
              -DefaultStorageContainerName $hadoopContainer' `
              -DataNodeVMSize $VmSize `
              -HeadNodeVMSize $HeadNodeVmSize

Upvotes: 2

Views: 847

Answers (3)

user5430505
user5430505

Reputation: 41

You can use the powershell command New-AzureRmHDInsightCluster to create cluster. and can provide value for -HeadNodeSize, -WorkerNodeSize. Follow link for VmSizes https://azure.microsoft.com/en-us/documentation/articles/cloud-services-sizes-specs/.

For D14 machine you can provide size as Standard_D14, mentioned in Size column in above article.

Upvotes: 0

Mikel Urkia
Mikel Urkia

Reputation: 2095

As you already mention, as for now, HDInsight clusters only support nodes within A3-A9 and D3-D14.

The link you provided shows pretty clear the codes asociated to each node type and size. The code Small belongs to the instance type A1\Small, which is not supported by HDInsight. Thus, it might consider the use of the smallest available instance, which is, in this case, A3\Large.

You should try specifying other instances that fall under the ranges supported by the HDInsight cluster. I can see you have already tried with extra large. You might want to try typing the code according to the code index available here (under Sizes for Web and Worker Role Instances), for example.

A3: Large
A4: Extralarge

Upvotes: 2

RickH
RickH

Reputation: 2486

Does it work if you use the values from the Sizes for Web and Worker Role Instances section like "ExtraLarge" (no space), "A6" or "Standard_D3"?

It looks like the node sizes that are currently supported are listed at: http://azure.microsoft.com/en-us/pricing/details/hdinsight/ but you have to use the id's from the Sizes for Web and Worker Role Instances section, not the Sizes for Virtual Machines section of https://msdn.microsoft.com/en-us/library/azure/dn197896.aspx.

Upvotes: 0

Related Questions