Terraform - Static ip addresses on Azure

Question

We have a requirement to configure static private ip's for the vm's that get deployed in Azure via terraform. Tjhe reason is that we then need to use these in Ansible via an ansible pipeline.

One solution I found here was to create a nic with a "dynamic" address first and then convert that to a "static" ip in the next step in Terraform.

# Create network interfaces with Private IP's
resource "azurerm_network_interface" "nic" {
  for_each = { for vm in var.vms : vm.hostname => vm }
  name                = "${each.value.hostname}-NIC"
  location            = var.network_location
  resource_group_name = var.vm_resource_group
  ip_configuration {
    name                          = "monitoringConfg"
    subnet_id                     = data.azurerm_subnet.vm_subnet.id
    private_ip_address_allocation = "dynamic"
  }
  tags = each.value.extra_tag
}

#Convert Dynamic Private IP's to Static
resource "azurerm_network_interface" "staticnic" {
  for_each = { for vm in var.vms : vm.hostname => vm }
  name                = "${each.value.hostname}-NIC"
  location            = var.network_location
  resource_group_name = var.vm_resource_group
  ip_configuration {
    name                          = "monitoringConfg"
    subnet_id                     = data.azurerm_subnet.vm_subnet.id
    private_ip_address_allocation = "static"
    private_ip_address            = azurerm_network_interface.nic[each.key].private_ip_address    
  }
  tags = each.value.extra_tag

But when I run this, I get the following error:

A resource with the ID "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxxxx/providers/Microsoft.Network/networkInterfaces/xxxxxxxxxxxxxxxxxxx-NIC" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_network_interface" for more information. on ../../modules/main.tf line 58, in resource "azurerm_network_interface" "staticnic": 58: resource "azurerm_network_interface" "staticnic" {

Does anyone have any idea what i am doing wrong or a better way to handle this?

Kind Regards, RB

Answer 1

I don't want to depend on local-execs. You can also use the azApi, docs on the Azure site too

What this basically does is do a rest-api call but with terraform credentials. Otherwise you still have to login with az login before you run terraform. Also not handy.

In our case we create a nic, then a virtual machine and then after the virtual machine is created we change the value of Dynamic to Static

An example piece for the azapi update:

resource "azapi_update_resource" "static_ip_linux" {
  count = var.os_flavor == "linux" ? 1 : 0
  type = "Microsoft.Network/networkInterfaces@2021-08-01"
  body = local.body_static_ip_encoded_json
  depends_on = [azurerm_linux_virtual_machine.linux_vm]
}

We have 2 flavors windows and linux hence the difference with count and depends on.

The body is created in the locals section and looks in our case like this:

  body_static_ip_encoded_json = jsonencode({
   "properties":{
      "ipConfigurations": [
        {
          "name": "${var.name}-ipconfig",
          "properties": {
            "privateIPAddress": "${azurerm_network_interface.nic[0].private_ip_addresses[0]}",
            "privateIPAllocationMethod": "Static",
            "subnet": {
              "id": data.azurerm_subnet.snet.id
            }
          }
        }
      ]
   },
   "location": "${var.location}"
  })

Don't forget to add to your versions.tf

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
    }
    azapi = {
      source = "azure/azapi"
    }
  }
}

This is needed so your module knows which provider to use. Otherwise if you terraform init it can complain about hashicorp/azapi (and it should be azure/azapi)

Read the docs for the default required_provider bit, which is the same as the default azurerm.

This works for all items that are not included yet with Terraform but do work via Rest API calls. (Azure preview for example, only when it is final will they usually update the rest/az cli commands)

Answer 2

So this seems to be a misconception about what "dynamic" means for a private IP in Azure. Creating a NIC in Azure with a "dynamic" private ip means the IP is assigned upon creation of the interface and only freed upon deletion of the interface.

This means it behaves exactly as "static" interfaces.

The only difference is that a "static" interface has a user assigned (as in input parameter) IP, a "dynamic" interface is automatically assigned a free IP from the subnet. I have send a PR to update the tf docs https://github.com/hashicorp/terraform-provider-azurerm/pull/15264

( From https://stackoverflow.com/a/70998327 )

Answer 3

For ease of reading this is what was done.

First, i created a service principle in azure using the steps listed here Creating a Service Principal

variables.TF

The command outputs were added to variables.TF as

variable APP_ID {
  default = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
}
variable SP_PASSWORD {
  default = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
variable TENANT_ID {
  default = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

This service principal variables will be used to do the az login before running the az network command on the windows worker agent.

Main.TF

resource "azurerm_network_interface" "nic" {
  for_each = { for vm in var.vms : vm.hostname => vm }
  name                = "${each.value.hostname}-NIC"
  location            = var.network_location
  resource_group_name = var.vm_resource_group
  ip_configuration {
    name                          = var.nic_ip_config
    subnet_id                     = data.azurerm_subnet.vm_subnet.id
    private_ip_address_allocation = "dynamic"
  }
  tags = each.value.extra_tag
}

To convert the above dynamic ip's tp static

# Convert All Private IP's from Dynamic to Static using powershell
resource "null_resource" "StaticIPsPrivateVMs" {
  for_each = { for vm in var.vms : vm.hostname => vm }
    provisioner "local-exec" {
      command = <<EOT
      az login --service-principal --username ${var.APP_ID} --password ${var.SP_PASSWORD} --tenant ${var.TENANT_ID}
      az network nic ip-config update -g ${var.vm_resource_group} --nic-name ${azurerm_network_interface.nic[each.key].name} --name ${var.nic_ip_config} --set privateIpAllocationMethod="Static"
      EOT
    interpreter = ["powershell", "-Command"]
  }
  depends_on = [
    azurerm_virtual_machine.vm
  ]
}

#AZ logout
resource "null_resource" "AzLogout" {
    provisioner "local-exec" {
      command = <<EOT
      az logout
      EOT
    interpreter = ["powershell", "-Command"]
  }
  depends_on = [
    null_resource.StaticIPsPrivateVMs
  ]
}

Azure-pipelines.yml

I added vmImage: 'windows-latest' at the top of the validate and deploy stages so that we don't end up using linux agents which will throw powershell not found errors:

  - stage: validate
    jobs:
    - job: validate
      pool:
        vmImage: 'windows-latest'
      continueOnError: false
      steps:

- stage: deploy
    jobs:
    - deployment: deploy
      pool:
        vmImage: 'windows-latest'
      continueOnError: false

Answer 4

Azure does not assign a Dynamic IP Address until the Network Interface is attached to a running Virtual Machine (or other resource), refer to this. So I think that we can't convert the Dynamic IP to the Static one before the VM created because the IP address does not exist for that time being.

Instead, we could directly associate some static IP addresses to the Azure VM by assigning some IP address in that subnet range. Read private IP allocation method.

Azure reserves the first four addresses in each subnet address range. The addresses can't be assigned to resources. For example, if the subnet's address range is 10.0.0.0/16, addresses 10.0.0.0-10.0.0.3 and 10.0.255.255 are unavailable.

For example, you may refer this template to configure static private ip's for the vms:

variable "vmlist" {
  type = map(object({
    hostname = string
    IP_address = string
  }))
  default = {
    vm1 ={
    hostname = "vma"
    IP_address = "10.0.2.4"
    },
    vm2 = {
    hostname = "vmb"
    IP_address = "10.0.2.5"
    }
  }
}

#...

resource "azurerm_network_interface" "staticnic" {
  for_each = var.vmlist
  name                = "${each.value.hostname}-nic"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name

  ip_configuration {
    name                          = "testconfiguration1"
    subnet_id                     = azurerm_subnet.internal.id
    private_ip_address_allocation = "Static"
    private_ip_address            = each.value.IP_address
  }
}

 #...

resource "azurerm_virtual_machine" "main" {
  for_each = var.vmlist
  name                  = each.value.hostname
  location              = azurerm_resource_group.main.location
  resource_group_name   = azurerm_resource_group.main.name
  network_interface_ids = [azurerm_network_interface.staticnic[each.key].id]
  vm_size               = "Standard_DS1_v2"

  # Uncomment this line to delete the OS disk automatically when deleting the VM
  # delete_os_disk_on_termination = true

  # Uncomment this line to delete the data disks automatically when deleting the VM
  # delete_data_disks_on_termination = true

  storage_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2016-Datacenter"
    version   = "latest"
  }

  storage_os_disk {
    name              = "${each.value.hostname}-osdisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }
  os_profile {
    computer_name  = each.value.hostname
    admin_username = "testadmin"
    admin_password = "Password1234!"
  }

   os_profile_windows_config {
    provision_vm_agent = "true"
  }

}

I am using

Terraform v0.14.7
+ provider registry.terraform.io/hashicorp/azurerm v2.52.0

Update

If you want to let Azure assign the dynamic IP and then convert it to a static one, you can use local-exec Provisioner to invoke a local executable after a resource is created.

resource "null_resource" "example" {

  for_each = var.vmlist
    provisioner "local-exec" {

   command = <<EOT

      $Nic = Get-AzNetworkInterface -ResourceGroupName ${azurerm_resource_group.main.name} -Name ${azurerm_network_interface.nic[each.key].name}
      $Nic.IpConfigurations[0].PrivateIpAllocationMethod = "Static"
      Set-AzNetworkInterface -NetworkInterface $Nic
   EOT
   
   interpreter = ["PowerShell", "-Command"]
  
  }
}