CODEWITHSUNDEEP
terraformvirtual-machinevspherepciterraform-provider-vsphere
noyro
noyro

Reputation: 1

vSphere error - Configured PCI devices are not attached to the VM

I'm trying to start few VMs via HashiCorp's Terraform (version 1.0.4) on vSphere 7.0, I configured PCI devices - Defined some PCI devices on ESXi host (direct-path passthrough devices), created a tf file with VM resource and pci_device_id parameter filled with a number of pci addresses, e.g. resource "vsphere_virtual_machine" "vm" { ... pci_device_id = ["0000:b5:00.0","0000:b5:00.1"] ...} as described above.
I got an error while deploying -
Error: A specified parameter was not correct: deviceChange[1].device.key on vm_plan_ldu.tf line 26, in resource "vsphere_virtual_machine" "vm_ldu": 26: resource "vsphere_virtual_machine" "vm_ldu" {

Eventually, none of PCI devices got attached to the VM as expected.

This is the configuration that I have used:

resource "vsphere_virtual_machine" "vm_udu" {
name                       = "cyrus2nr-udu1"
resource_pool_id           = data.vsphere_resource_pool.pool.id
datastore_id               = data.vsphere_datastore.datastore.id
host_system_id             = data.vsphere_host.host.id
num_cpus = var.num_cpus_udu1
num_cores_per_socket = var.num_cpus_udu1
cpu_reservation = var.host_cpu_freq == "" ? (var.auto_host_cpu_freq * var.num_cpus_udu1) : (var.host_cpu_freq * 
var.num_cpus_udu1)
cpu_share_level = "high"

memory   = var.memory_udu1
memory_reservation = var.memory_udu1
memory_share_level = "high"

guest_id = data.vsphere_virtual_machine.template_udu.guest_id
pci_device_id = ["0000:5e:0e.0","0000:5e:0e.1","0000:5e:0a.0"]

scsi_type = data.vsphere_virtual_machine.template_udu.scsi_type

network_interface {
    network_id   = data.vsphere_network.network_udu_1.id
    adapter_type = data.vsphere_virtual_machine.template_udu.network_interface_types[0]
}

disk {
    label            = "disk0.vmdk"
    size             = data.vsphere_virtual_machine.template_udu.disks.0.size
    eagerly_scrub    = data.vsphere_virtual_machine.template_udu.disks.0.eagerly_scrub
    thin_provisioned = data.vsphere_virtual_machine.template_udu.disks.0.thin_provisioned
    io_limit         = 1000
}
disk {
    label       = "disk1"
    size        = "48"
    unit_number = 1
    thin_provisioned = false
    io_limit    = 1000
}

clone {
    template_uuid = data.vsphere_virtual_machine.template_udu.id
}

cdrom {
    client_device = true
}

// Attach the cloudcongif as a OVF property "user-data"
vapp {
    properties = {
        "user-data" = base64encode(local_file.cloudconfig-udu.content)
    }
  }
}

Any ideas how to deal with it? Thanks!

Upvotes: 0

Views: 770

Answers (1)

Erik
Erik

Reputation: 107

I am unfamiliar with HashiCorp's Terraform product but from your description it sounds like there's a significant chance that you did not configure your VMs to NOT be vMotionable. Perhaps Terraform takes care of this for you or perhaps there is now an infrastructure to allow vMotioning of VMs with passthrough of host PCI devices. There is one for host USB devices that are passed through to VMs but I'm unaware of a corresponding support for vMotion of VMs with passthrough of PCI devices.

To see the issues involved see https://kb.vmware.com/s/article/1022290?lang=en_US&queryTerm=usb%20arbitrator and follow the link in this text:

Note: To support vMotion for Host-Connected devices, select Support vMotion while device is connected. For more information, see the Configuring USB Passthrough Devices for vMotion section in the vSphere Virtual Machine Administration Guide.

Here's a post for how to disable vMotion: https://communities.vmware.com/t5/vMotion-Resource-Management/Disable-vMotion-for-a-single-VM/td-p/435730. VMware doesn't encourage it because it defeats the purpose of DRS, HA and a bunch of their other products. Back in the day I was part of the team that enabled USB passthrough and to release the new feature it was a requirement that we solve the "vMotion problem" which is why there is a "USB arbitrator", so perhaps there is now one for PCI devices as well.

Upvotes: 0

Related Questions