How to add a disk to VM when cloning from a template?

Question

I've found pyvmomi examples on how to add a disk to an already existing VM, but I would like to customize the VM template and then clone. Setting the CPUs and memory are pretty straight forward, but the adding of one or more disks to an existing template, asides from the boot disk, eludes me.

# Add an additional 200 GB disk
new_disk_kb = int(20) * 1024 * 1024
disk_spec = vim.vm.device.VirtualDeviceSpec()
disk_spec.fileOperation = "create"
disk_spec.operation = vim.vm.device.VirtualDeviceSpec.Operation.add
disk_spec.device = vim.vm.device.VirtualDisk()
disk_spec.device.backing = vim.vm.device.VirtualDisk.RawDiskMappingVer1BackingInfo()
disk_spec.device.backing.diskMode = 'persistent'
disk_spec.device.unitNumber = 3
disk_spec.device.capacityInKB = new_disk_kb

# vm configuration
vmconf = vim.vm.ConfigSpec()
vmconf.numCPUs = 8            # change the template's cpus from 4 to 8
vmconf.memoryMB = 16 * 1024   # change the template's memory from 4 GB to 16 GB
# change the template's disks from
#    1 250 GB boot, 1 x 200 GB disk
# to 
#    1 250 gB boot, 2 x 200 GB disks
vmconf.deviceChange = [ disk_spec ]  # something is not right
                               
clonespec = vim.vm.CloneSpec()
clonespec.location = relospec
clonespec.powerOn = True
clonespec.config = vmconf
clonespec.customization = customspec
task = template.Clone(folder = destfolder, name = vmname, spec = clonespec)

The code works without the vmconf.deviceChange. Once I try to add a disk I see the error

Invalid configuration for device '0'.

or

Incompatible device backing specified for device '0'.

Answer 1

To determine the proper unit number to assign I use the following code.

# Choose a controller with the least number of devices on it
chosenController = None
for controller in self.getParavirtualScsiControllers(vm):
    if chosenController == None:
        chosenController = controller
    else:
        if len(chosenController.device) > len(controller.device):
            chosenController = controller

if chosenController== None:
    raise SystemExit("No SCSI controller. Cannot add LUNs")

# Get a list of used unit ids
unitidList = []
for unit in chosenController.device:
    for dev in vm.config.hardware.device:
        if dev.key == unit and isinstance(dev, vim.vm.device.VirtualDisk):
            unitidList.append(dev.unitNumber)

if len(unitidList) == 0:    # If none found use 0
    unitNumber = 0
else:                       # otherwise id the lowest unused id
    for i in list(range(0,7))+list(range(8,17)): # skip 7 it is reserved for the iscsi controller
                                                 # 16 is illegal but checked for below
        if i not in unitidList:
            unitNumber = i
            break
                
if unitNumber > 15: 
    raise SystemExit("No SCSI controller with less than 16 LUNS. Add more SCSI controllers")

Answer 2

I just ran into this exact issue adding RDM disks to a vm.

The problem is:

disk_spec.device.backing = vim.vm.device.VirtualDisk.RawDiskMappingVer1BackingInfo()

You need to provide backing.deviceName and backing.lunUuid to your spec. After that it'll work. I am confident because I saw the exact same error message and it fixed it for me.

You get the lun uuid and deviceName from host.config.storageDevice.scsiLun for the lun you're interested in.

Answer 3

The problem seem to be in the following part of the code:

# Add an additional 200 GB disk
new_disk_kb = int(20) * 1024 * 1024
disk_spec = vim.vm.device.VirtualDeviceSpec()
disk_spec.fileOperation = "create"
disk_spec.operation = vim.vm.device.VirtualDeviceSpec.Operation.add
disk_spec.device = vim.vm.device.VirtualDisk()
disk_spec.device.backing = vim.vm.device.VirtualDisk.RawDiskMappingVer1BackingInfo()
disk_spec.device.backing.diskMode = 'persistent'
disk_spec.device.unitNumber = 2
disk_spec.device.capacityInKB = new_disk_kb

If the VM already has a disk, the boot disk, then the unitNumber should start at 2.

Answer 4

This question has multiple moving parts including creating a template, modifying a template and cloning the VM.

Let's tackle the easiest issue first. This part of your code is incorrect:

new_disk_kb = int(20) * 1024 * 1024
disk_spec = vim.vm.device.VirtualDeviceSpec()
disk_spec.fileOperation = "create"
disk_spec.operation = vim.vm.device.VirtualDeviceSpec.Operation.add
disk_spec.device = vim.vm.device.VirtualDisk()
disk_spec.device.backing = vim.vm.device.VirtualDisk.RawDiskMappingVer1BackingInfo()
disk_spec.device.backing.diskMode = 'persistent'
disk_spec.device.unitNumber = 3
disk_spec.device.capacityInKB = new_disk_kb

After reviewing multiple pyvmomi samples scripts I can see that you're not setting the unitNumber correctly. Based on these scripts you need to determine the number of disks and assign the next unitNumber available, which I would assume in your case is 2 and not 3.

The code below was extracted from the add raw disk to vm pyvmomi script.

spec = vim.vm.ConfigSpec()
# get all disks on a VM, set unit_number to the next available
unit_number = 0
controller = None
for device in vm.config.hardware.device:
    if hasattr(device.backing, 'fileName'):
        unit_number = int(device.unitNumber) + 1
        # unit_number 7 reserved for scsi controller
        if unit_number == 7:
            unit_number += 1
        if unit_number >= 16:
            print("we don't support this many disks")
            return -1
    if isinstance(device, vim.vm.device.VirtualSCSIController):
        controller = device
if controller is None:
    print("Disk SCSI controller not found!")
    return -1
disk_spec = vim.vm.device.VirtualDeviceSpec()
disk_spec.fileOperation = "create"
disk_spec.operation = vim.vm.device.VirtualDeviceSpec.Operation.add
disk_spec.device = vim.vm.device.VirtualDisk()
rdm_info = vim.vm.device.VirtualDisk.RawDiskMappingVer1BackingInfo()
disk_spec.device.backing = rdm_info
disk_spec.device.backing.compatibilityMode = disk_compatibility_mode
disk_spec.device.backing.diskMode = disk_mode
# The device_name will look something like
#     /vmfs/devices/disks/naa.41412340757396001d7710df0fdd22a9
disk_spec.device.backing.deviceName = device_name
disk_spec.device.unitNumber = unit_number
disk_spec.device.controllerKey = controller.key
spec.deviceChange = [disk_spec]

Concerning the other problems that you want to tackle. There are several pyvmomi script that could be useful. One in particular is Clone VM, which provides details on cloning a VM from a template, which you're trying to create.

I'm not sure how useful that script would be, because it seems that you're looking for a complete solution to solve your use case.

I would recommend looking at this old Github project, which has lots of useful code for creating a template, modifying a template and cloning a VM.

Concerning your comments about using an existing template that already has one disk (boot disk) and adding a secondary disk during the cloning process.

Based on my interpretation on the VM Documentation I currently don't think this is possible and where is why:

1. A VM template is a master copy image of a virtual machine that includes VM disks, virtual devices, and settings. This template can be use in the VM cloning process. This template is static and should be considered a configuration baseline.

2. This template cannot be modify during the cloning process. If this static template needs to be modified for any reason than it needs to be recreated with the new additions. This means that the original template needs to be converted into a VM, this VM needs to be edited, and then convert back into a new template.

If this isn't possible you can clone the baseline template on a new VM and use one of the pyvmomi add disks scripts to enable a new disk on your VM.

I don't know your exact use case, but I think recreating the template is the logical choice, but your use case might work better with creating the new disk after cloning.

That GitHub project that I previously mentioned is using a baseline template for cloning a VM and it also has an addDisks function. The latter is used to add a new disk to the VM that was created during the cloning process.

Here is an older pyvmomi issue where someone wanted to modify a disk related to a template being cloned. It was recommended that the OP perform this task in the post-clone process.

That GitHub project shows how to use a baseline template and modify the VMs configuration as needed. From the code it seems that it is using CloneSpec() to modify the VM during the cloning process and not the post-clone process.

Answer 5

I am not familiar with the pyvmomi tool, but it seems to me that the disk ought to be attached to a controller. The UnitNumber makes no sense if there is no controller to supply the unit.

When associating virtual devices with default devices, the controllerKey property should be reset with the key property of the controller [ here ]

Can you try the above association? (Sorry, I am not quite sure if you should use vmconf or clonespec.config)

controller = next( \
 (device for device in vmconf.hardware.device \
     if isinstance(device, vim.vm.device.VirtualSCSIController) \
 ), None)
disk_spec.device.controllerKey = controller.key

Answer 6

The errors which you're getting seem to indicate that you need to fix the unit_number. Keep in mind that 7 is reserved for the scsi controller and 16 is the maximum supported number of disks by VMWare (view sample code).