Vagrant is apparently not merging Vagrantfiles correctly, Chef config is lost

Question

First off here's the error I'm getting:

There are errors in the configuration of this machine. Please fix
the following errors and try again:

chef client provisioner:
* Chef server URL must be populated.
* Validation key path must be valid path to your chef server validation key.

The chef server URL does appear to be populated, and the validation key path is valid.

I've got 3 Vagrantfiles in effect, and I'm using the documentation to try to get the order correct.

This works fine when I stick everything in one Vagrantfile in the project directory, but I want to set defaults and not have to copy paste everything.

1) The Vagrantfile packaged with the box:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.provider :vsphere do |vsphere|
    vsphere.host = 'vsphereserver.example.com'
    vsphere.compute_resource_name = 'TestDev'
    vsphere.user = 'vagrantadmin'
    vsphere.password = 'password'
    vsphere.insecure = true
  end

  config.ssh.username = 'auto'
  config.ssh.private_key_path = '~/.vagrant.d/id_rsa'
end

2) The Vagrantfile in my home directory (~/.vagrant.d):

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = 'vsphere'

  config.vm.provider :vsphere do |vsphere|
    vsphere.template_name = 'vagrantchefnode'
  end

  config.vm.provision "chef_client" do |chef|
    chef.add_role "base"
    chef.provisioning_path = "/etc/chef"
    chef.chef_server_url = "https://chefserver.example.com"
    chef.validation_key_path = "/home/user/.vagrant.d/chef/validation.pem"
    chef.client_key_path = "/etc/chef/client.pem"
    chef.validation_client_name = "chef-validator"
    chef.custom_config_path = "/home/user/.vagrant.d/Vagrantfile.chef"
    chef.delete_node = true
    chef.delete_client = true
  end
end

3) Vagrantfile from the project directory (~/.vagrant.d/boxes/chefnode1):

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.provider :vsphere do |vsphere|
#    vsphere.template_name = 'chefnode'
    vsphere.customization_spec_name = 'chefnode1'
    vsphere.name = 'chefnode1'
  end

  config.vm.provision "chef_client" do |chef|
    chef.node_name = "chefnode1"
    chef.add_role "test"
  end

end

Is it overwriting the entire chef_client configuration somehow? If so, how do I make it stick and merge the way it's supposed to?

Update:

I've got something working with Tejay's answer and this doc, but it requires translating to a different syntax, and I can't figure out how to get methods like add_role translated. Here's what I've got:

2)

  config.vm.provision "chef_client",
    id: "chef",
    provisioning_path: "/etc/chef",
    chef_server_url: "https://chefserver.example.com",
    validation_key_path: "/home/user/.vagrant.d/chef/validation.pem",
    client_key_path: "/etc/chef/client.pem",
    validation_client_name: "chef-validator",
    custom_config_path: "/home/user/.vagrant.d/Vagrantfile.chef",
    delete_node: true,
    delete_client: true

3)

  config.vm.provision "chef_client",
    id: "chef",
    node_name: "chefnode1"

So this brings the machine up but I can't specify a run_list.

Answer 1

You are actually creating two provisioners, one in each file, and so the provisioner in your project directory does not have a URL or validation key. Vagrant's documentation explains that you need to use an id in order to modify an existing provider. Take a look at the section "OVERRIDING PROVISIONER SETTINGS" on this page:

https://docs.vagrantup.com/v2/provisioning/basic_usage.html

So try this (not tested):

The Vagrantfile in my home directory (~/.vagrant.d):

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = 'vsphere'

  config.vm.provider :vsphere do |vsphere|
    vsphere.template_name = 'vagrantchefnode'
  end

  config.vm.provision("chef_client", id: 'someID') do |chef|
    chef.add_role "base"
    chef.provisioning_path = "/etc/chef"
    chef.chef_server_url = "https://chefserver.example.com"
    chef.validation_key_path = "/home/user/.vagrant.d/chef/validation.pem"
    chef.client_key_path = "/etc/chef/client.pem"
    chef.validation_client_name = "chef-validator"
    chef.custom_config_path = "/home/user/.vagrant.d/Vagrantfile.chef"
    chef.delete_node = true
    chef.delete_client = true
  end
end

Vagrantfile from the project directory (~/.vagrant.d/boxes/chefnode1):

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.provider :vsphere do |vsphere|
#    vsphere.template_name = 'chefnode'
    vsphere.customization_spec_name = 'chefnode1'
    vsphere.name = 'chefnode1'
  end

  config.vm.provision("chef_client", id: 'someID') do |chef|
    chef.node_name = "chefnode1"
    chef.add_role "test"
  end
end