Chef install series of pre-packaged gems

Question

I am trying to install a series of gems required by my recipes but I would rather not install the development tools (i.e. gcc, gcc-c++, ruby-devel. see this for more context) on my production machines, so instead have included the relevant gem files as resources in my cookbook.

I am using this code

node["chef_gems"].each do |pkg, attrs|
  gem_file = "#{pkg}-#{attrs["version"]}.gem"
  dest_gem_file = "#{Chef::Config['file_cache_path']}/#{gem_file}"
  cookbook_file "#{dest_gem_file}" do
      source "gems/#{gem_file}"
  end
  if File.exist?("#{dest_gem_file}")
    chef_gem pkg do
      source "#{dest_gem_file}"
      action :install
    end
  end
end

Issues I am running in to

1. Currently this requires two chef runs because of the File.exist? check but without it, it fails to compile because the file doesn't exist during the compilation phase. Is there a simpler way to accomplish this?
2. Some of these gems have dependencies so ordering matters which seems to default to alphabetical. I am about to go down the path of enforcing order via a sort ordering key (ugly), was wondering if there is a better way.
3. Stepping back, this seems overly complicated. Is there a simpler strategy to achieve the goal of installing the gems and their dependencies without installing development tools on production servers.

Here is the original error without the File.exist? check

Starting Chef Client, version 11.4.0
resolving cookbooks for run list: ["cis-rhel", "ec2-hostname-rhel", "chef-client", "cassandra"]
Synchronizing Cookbooks:
<REDACTED>
Compiling Cookbooks...
[2013-10-16T18:45:01-04:00] WARN: Cloning resource attributes for user[<REDACTED>] from prior resource (CHEF-3694)
<REDACTED>
[2013-10-16T18:45:01-04:00] WARN: Current  user[cassandra]: /var/chef/cache/cookbooks/cassandra/recipes/user.rb:23:in `from_file'
Recipe: cassandra::packages
  * chef_gem[cassandra-cql] action install
================================================================================
Error executing action `install` on resource 'chef_gem[cassandra-cql]'
================================================================================


Gem::Exception
--------------
Cannot load gem at [/var/chef/cache/cassandra-cql-1.1.4.gem] in /


Cookbook Trace:
---------------
/var/chef/cache/cookbooks/cassandra/recipes/packages.rb:36:in `block in from_file'
/var/chef/cache/cookbooks/cassandra/recipes/packages.rb:29:in `each'
/var/chef/cache/cookbooks/cassandra/recipes/packages.rb:29:in `from_file'
/var/chef/cache/cookbooks/cassandra/recipes/default.rb:23:in `from_file'


Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/cassandra/recipes/packages.rb

 36:     chef_gem pkg do
 37:       source "#{dest_gem_file}"
 38:       action :install
 39:     end
 40: #  end



Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/cassandra/recipes/packages.rb:36:in `block in from_file'

chef_gem("cassandra-cql") do
  provider Chef::Provider::Package::Rubygems
  action [:install]
  retries 0
  retry_delay 2
  package_name "cassandra-cql"
  source "/var/chef/cache/cassandra-cql-1.1.4.gem"
  cookbook_name "cassandra"
  recipe_name "packages"
end




================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/cassandra/recipes/default.rb
================================================================================


Gem::Exception
--------------
chef_gem[cassandra-cql] (cassandra::packages line 36) had an error: Gem::Exception: Cannot load gem at [/var/chef/cache/cassandra-cql-1.1.4.gem] in /


Cookbook Trace:
---------------
  /var/chef/cache/cookbooks/cassandra/recipes/packages.rb:36:in `block in from_file'
  /var/chef/cache/cookbooks/cassandra/recipes/packages.rb:29:in `each'
  /var/chef/cache/cookbooks/cassandra/recipes/packages.rb:29:in `from_file'
  /var/chef/cache/cookbooks/cassandra/recipes/default.rb:23:in `from_file'


Relevant File Content:
----------------------
/var/chef/cache/cookbooks/cassandra/recipes/packages.rb:

 29:  node["cassandra"]["chef_gems"].each do |pkg, attrs|
 30:    gem_file = "#{pkg}-#{attrs["version"]}.gem"
 31:    dest_gem_file = "#{Chef::Config['file_cache_path']}/#{gem_file}"
 32:    cookbook_file "#{dest_gem_file}" do
 33:        source "gems/#{gem_file}"
 34:    end
 35:  #  if File.exist?("#{dest_gem_file}")
 36>>     chef_gem pkg do
 37:        source "#{dest_gem_file}"
 38:        action :install
 39:      end
 40:  #  end
 41:  end
 42:  
 43:  # Some distributed packages of Cassandra start the service in their
 44:  # postinstall; keep them all equal and a restart can be done after the configs
 45:  # are written on the first run. Added difficulty: they also come with init



[2013-10-16T18:45:01-04:00] ERROR: Running exception handlers
[2013-10-16T18:45:02-04:00] FATAL: Saving node information to /var/chef/cache/failed-run-data.json
[2013-10-16T18:45:02-04:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated
[2013-10-16T18:45:02-04:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2013-10-16T18:45:02-04:00] FATAL: Gem::Exception: chef_gem[cassandra-cql] (cassandra::packages line 36) had an error: Gem::Exception: Cannot load gem at [/var/chef/cache/cassandra-cql-1.1.4.gem] in /

Answer 1

1. You can and should skip the if File.exist?("#{dest_gem_file}") segment, as the previous resource will always create the file or raise an exception.
2. Installing these with a package manager that can deal with dependencies may be a much better option, if you can do that.
3. If it's not possible to do it with the rubygems package manager because it needs to compile native extensions for packages, you could consider packaging the compiled versions and using the systems package manager to install them.