GitLab Runner Parallels Executor & Windows 10 VM - Stuck "Creating new VM..."

Question

Summary

Hi,

I originally posted an issue of what I believe to be a bug with GitLab Runner, but have sadly received no response on the GitLab-Runner repository.

I'm using Parallels Executor in order to test/build/deploy an Electron+React application using a pre-prepared MacOS and Windows 10 VM which also have SSH access enabled (tested too).

When registering the runners I then proceeded to add the relevant SSH user and password details into the config.toml.

The MacOS image works fine when a new build comes through, pipeline succeeds. However, the Windows 10 stages just hang endlessly and times out:
Screenshot of error in the Runner pipeline

The Runner receives the job and Parallels even creates the template VM and the linked image. It appears as though GitLab Runner doesn't receive confirmation of this though and proceeds to do nothing which leads to the job timing out after a number of reattempts.

Hard to know whether there's something intricate I am missing with Windows, or whether this might be a bug with Gitlab Runner itself.

Any ideas?

Steps to reproduce

• Create a Windows 10 Parallels VM and install/configure OpenSSH
• Register a runner using Parallels as the executor
• Edit the config.toml and add the SSH user and password under the [runners.ssh] section
• Ensure the Windows 10 VM image is properly shut down (not suspended)
• Run a build and watch the above happen

Gitlab-ci yaml file

stages:
  - test

test-win10:
    stage: test
    tags:
    - windows10
    variables:
      NODE_ENV: "development"
      DEBUG_MODE: 'true'
    script:
      - npm install
      - npm run test

Actual behavior

The Windows 10 VM will be started by Parallels, based off the template it created. From there, the build process will not proceed any further and you'll be left with a Windows 10 VM running happily doing 0. An hour later the build will fail.

Expected behavior

The defined stage should complete!

Runner Output Log

1 Running with gitlab-runner 12.6.0 (ac8e767a)
2   on [Ravs iMac]Parallels Windows 10 VM rpMNC3Qo
3
Using Parallels 14.1.0 executor...
4 Creating new VM...
5 ERROR: Preparation failed: PrlctlOutput error: 
6 Will be retried in 3s ...
7 Using Parallels 14.1.0 executor...
8 Creating new VM...
9 ERROR: Preparation failed: PrlctlOutput error: 
10 Will be retried in 3s ...
11 Using Parallels 14.1.0 executor...
12 Creating new VM...
13 ERROR: Job failed: execution took longer than 1h0m0s seconds

Config.toml

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "[Ravs iMac]Parallels MacOS VM"
  url = "https://gitlab.com/"
  token = "**********"
  executor = "parallels"
  [runners.custom_build_dir]
  [runners.ssh]
    user = "ci"
    password = "password"
    host = "macos-ci.shared"
    port = "22"
  [runners.parallels]
    base_name = "macOS_CI"
    disable_snapshots = false
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

[[runners]]
  name = "[Ravs iMac]Parallels Windows 10 VM"
  url = "https://gitlab.com"
  token = "**********"
  executor = "parallels"
  [runners.custom_build_dir]
  [runners.ssh]
    user = "ci"
    password = "password"
    host = "10.211.55.5"
    port = "22"
  [runners.parallels]
    base_name = "Windows10_CI"
    disable_snapshots = false
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Used GitLab Runner version

Version:      12.6.0
Git revision: ac8e767a
Git branch:   12-6-stable
GO version:   go1.13.4
Built:        2019-12-22T11:55:34+0000
OS/Arch:      darwin/amd64

Parallels Version

Version 14.1.0 (45387) Business Edition

Possible fixes

N/A

Answer 1

The 2nd poster mentioned an IP lease not found for MAC address. I believe this is a different issue. I have chanced upon this as well. I solved it by first killing all instances of the gitlab runner. I then emptied this file: /Library/Preferences/Parallels/parallels_dhcp_leases. (Make a copy before you do that just in case, of course)

This requires root access.

Once I did that, all instances of the VM were able to be created without issues.

Answer 2

I see the OP is having a prlctl issue. I think I know what this is. If you have multiple instances of the gitlab runner software running, you can get prlctl errors. By default, the gitlab runner configures itself to run on bootup. If you then run "gitlab-runner run", you launch a 2nd instance and the two don't play well in the same sandbox.

To remedy, kill all known instances, then run "gitlab-runner stop". After that, "gitlab-runner status" should show you a goofy error:

Could not find service "gitlab-runner" in domain for port

You're on the right track! Now just launch a single instance manually and things should come back to normal.

Answer 3

I've been having this problem for years. Just tried a new gitlab-runner setup (with gitlab-runner on the Mac and using Win10 under Parallels) and it doesn't work.

There's a tell-tale error in the gitlab-runner output:

ERROR: Job failed (system failure): IP lease not found for MAC address 001C42C13862 in: /Library/Preferences/Parallels/parallels_dhcp_leases  duration=6m40.688084247s job=1125551877 project=25368177 runner=xz2gps

I traced that (last year) and it turns out that the cloning of the VM gives it a new MAC address, and gitlab-runner looks for the old MAC address (of the original VM, not the clone). So gitlab-runner thinks the VM hasn't started.

The weird thing is that I have exactly the same setup with a MacOS Parallels VM and with that gitlab-runner has no problem at all.

I've been hunting for a solution for 2 years with no result. Mainly answering the old thread here in the hope that someone someday will find it and post a solution.

[Update]

I may have found a workaround. The problem seems to be that the cloned Windows VM uses the old MAC address (from the clone parent), not the new one. (So that's the reverse of what I suggested in the previous answer).

The workaround is that after I have built the Windows VM and shut it down and before I ever run gitlab-runner I change the Parallels network configuration for the parent VM and give it a new MAC address and a different type of network card. And not ever run it again. The clone will then come up and see a new network card, and gets the correct new MAC address.

It remains to be seen if this is a long term solution...