CODEWITHSUNDEEP
chef-infra
Chris F
Chris F

Reputation: 16812

How do I manage a single chef server to be used by all

I'm past being new to Chef, though NOT an expert by any means. Using open-source Chef, I'm setting up a single chef server that will have ALL my company's cookbooks that aim to install our product application on a VMs that's running a base OS, say RHEL 5.10. I'm segregating the cookbook files in version control, but how do I segregate them when I funnel them in the ONE chef server? Here are my challenges:

Here's a simple work flow, extending my ActiveMQ I alluded to above

I know this may be a loaded post. Is there a better forum for this? Can someone point me to a link where pieces of my scenario are done? Thanks.

Upvotes: 1

Views: 494

Answers (2)

Mark O'Connor
Mark O'Connor

Reputation: 78021

When running chef at scale there are three ways to isolate configuration between users.

  1. Use enterprise chef and setup an chef "organisation" for functional group of users
  2. Use open source chef server and setup a server for each functional group of users.
  3. Create chef "environments" for each functional group, within a single instance of chef server.

I would love to see more discussion or publications from users operating chef at scale. Some of us need to support dev teams who do not use chef and cannot live with a one size fits all server setup.

Options 1 + 2

In practice, I have discovered that enterprise chef "organisations" are functionally the same as running separate servers. You still need to load cookbooks, roles, data bags, separately into each.

I maintain this approach is simple to understand but complex to maintain. Each instance of chef server/organisation requires its own chef repo, access keys and Jenkins loading jobs.

Option 3

Using chef environments is harder to understand. Once it is properly understood I think it's simpler to implement. Berkshelf is tool that makes it all possible.

First of all don't be mislead by the chef documentation. The examples of environments they give are global in nature:

  • Development
  • Test
  • Production

This gives the impression that all production applications would share the same set of cookbooks. Once the number of apps starts to increase, you start getting into lots of trouble with conflicting run-times. Berkshelf v2.0 really sucked here... It frequently loaded inconsistent sets of cookbooks with absolutely no warning :-(

So.. Berkshelf v3.0 to the rescue and Jamie Winsor's blog posting on cookbook patterns:

The Environment cookbook pattern proposes a chef environment for each application instance.

  • app1-dev
  • app1-test
  • app1-prod

The "Berkshelf.lock" is committed and saved into the Environment cookbook version and used to load the cookbook dependencies of a particular environment. An example of this pattern in action is the berkshelf-api installation.

The lightbulb for me was then I found the following chef search condition in the haproxy cookbook

pool_members = search("node", "role:#{node['haproxy']['app_server_role']} AND chef_environment:#{node.chef_environment}") || []

The members of the load balanced pool share a common role and environment. So... Either the role is application instance specific or the environment is!

Upvotes: 3

coderanger
coderanger

Reputation: 54267

As mentioned up in the comments, this is generally solved through the use of library and wrapper cookbooks. You would have a shared activemq cookbook (possibly from the community site or written in-house) which provides a the core configuration steps for ActiveMQ. The nfor each environment you would have a cookbook/recipe like prod_a::activemq or which sets whatever specific attributes or other things you need for that environment and then either uses include_recipe or LWRPs from the activemq cookbook. You can find more information about this environment cookbook pattern on Jamie's blog. You can also check out poise-appenv if you want an intermediate layer of environment-y config. You can use knife-solve to always see which versions of each cookbook are active on a given node.

Upvotes: 1

Related Questions