OpsWorks: Cookbook httpd not found on recipe run

Question

I've just started working with Chef and OpsWorks on AWS in an effort to improve our infrastructure.

I'm running into some problems when attempting to run a recipe that I've set up to do (I think) some pretty basic things (setting up httpd and such).

I keep running into problems with including recipes though before the configuration stuff actually gets to run.

Here is the error I am getting:

================================================================================
Recipe Compile Error in /var/lib/aws/opsworks/cache.stage2/cookbooks/Tokeniser/recipes/default.rb
================================================================================


Chef::Exceptions::CookbookNotFound
----------------------------------
Cookbook httpd not found. If you're loading httpd from another cookbook, make sure you configure the dependency in your metadata


Cookbook Trace:
---------------
/var/lib/aws/opsworks/cache.stage2/cookbooks/Tokeniser/recipes/default.rb:12:in `from_file'


Relevant File Content:
----------------------
/var/lib/aws/opsworks/cache.stage2/cookbooks/Tokeniser/recipes/default.rb:

5:  # Copyright (C) 2016 YOUR_NAME
6:  #
7:  # All rights reserved - Do Not Redistribute
8:  #
9:  
10:  include_recipe("deploy")
11:  include_recipe("php")
12>> include_recipe("httpd")
13:  
14:  package 'apache2' do
15:    case node[:platform]
16:    when 'centos','redhat','fedora','amazon'
17:      package_name 'httpd'
18:    when 'debian','ubuntu'
19:      package_name 'apache2'
20:    end
21:    action :install

This is after I already have a berksfile in my root that consists of:

source "https://supermarket.chef.io"

metadata

cookbook 'httpd', '~> 0.3.3'
cookbook 'php', '~> 1.7.2'
cookbook 'magic_shell', '~> 1.0.0'
cookbook 'composer', '~> 2.2.1'

The failure log even notes the httpd recipe being fetched:

[2016-01-20T16:18:17+00:00] INFO: 
Resolving cookbook dependencies...
Fetching 'Tokeniser' from source at .
Fetching cookbook index from https://supermarket.chef.io...
Using Tokeniser (0.1.0) from source at .
Installing build-essential (2.2.4) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing chef-sugar (3.3.0) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing chef_handler (1.2.0) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing compat_resource (12.5.26) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing composer (2.2.1) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing httpd (0.3.3) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing iis (4.1.5) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing magic_shell (1.0.0) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing mysql (6.1.2) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing php (1.7.2) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing rbac (1.0.3) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing smf (2.2.7) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing windows (1.39.1) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing xml (2.0.0) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing yum (3.9.0) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing yum-epel (0.6.5) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Installing yum-mysql-community (0.1.21) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io:443/api/v1)
Vendoring Tokeniser (0.1.0) to /opt/aws/opsworks/current/berkshelf-cookbooks/Tokeniser
Vendoring build-essential (2.2.4) to /opt/aws/opsworks/current/berkshelf-cookbooks/build-essential
Vendoring chef-sugar (3.3.0) to /opt/aws/opsworks/current/berkshelf-cookbooks/chef-sugar
Vendoring chef_handler (1.2.0) to /opt/aws/opsworks/current/berkshelf-cookbooks/chef_handler
Vendoring compat_resource (12.5.26) to /opt/aws/opsworks/current/berkshelf-cookbooks/compat_resource
Vendoring composer (2.2.1) to /opt/aws/opsworks/current/berkshelf-cookbooks/composer
Vendoring httpd (0.3.3) to /opt/aws/opsworks/current/berkshelf-cookbooks/httpd
Vendoring iis (4.1.5) to /opt/aws/opsworks/current/berkshelf-cookbooks/iis
Vendoring magic_shell (1.0.0) to /opt/aws/opsworks/current/berkshelf-cookbooks/magic_shell
Vendoring mysql (6.1.2) to /opt/aws/opsworks/current/berkshelf-cookbooks/mysql
Vendoring php (1.7.2) to /opt/aws/opsworks/current/berkshelf-cookbooks/php
Vendoring rbac (1.0.3) to /opt/aws/opsworks/current/berkshelf-cookbooks/rbac
Vendoring smf (2.2.7) to /opt/aws/opsworks/current/berkshelf-cookbooks/smf
Vendoring windows (1.39.1) to /opt/aws/opsworks/current/berkshelf-cookbooks/windows
Vendoring xml (2.0.0) to /opt/aws/opsworks/current/berkshelf-cookbooks/xml
Vendoring yum (3.9.0) to /opt/aws/opsworks/current/berkshelf-cookbooks/yum
Vendoring yum-epel (0.6.5) to /opt/aws/opsworks/current/berkshelf-cookbooks/yum-epel
Vendoring yum-mysql-community (0.1.21) to /opt/aws/opsworks/current/berkshelf-cookbooks/yum-mysql-community

And as you can see php is also required but it doesn't error there.

I can't see what I'm doing wrong. I'm probably being an idiot. Any help or ideas appreciated.

Answer 1

So I finally found the answer to this was that I was running on OpsWorks' Chef 12 version, which doesn't use Berkshelf on the server and requires you to package you cookbooks using berks package, rather than loading from a git repo, unless you only have one cookbook or don't use berkshelf to manage dependancies.

The Amazon documentation is tricky to find but is here: http://docs.aws.amazon.com/opsworks/latest/userguide/best-practices-packaging-cookbooks-locally.html

Answer 2

IIUYC you need to add the http cookbook dependency to the Tokeniser cookbook metadata.rb file, not to the Berksfile:

# Tokeniser/metadata.rb

 depends 'httpd', '~> 0.3.3'

The same applies to the php and deploy cookbooks, remember to put them in the metadata file.