CODEWITHSUNDEEP
performancechef-infrachef-recipechef-solo
roo2
roo2

Reputation: 6071

why is chef-client synchronization slow for local cookbooks

Background

I'm running chef-client --local-mode with cookbook path set to ./berks-cookbooks these cookbooks where already downloaded by berkshelf before running chef. (chef sdk 4.0.0, berkshelf 3.2.3)

Before running my scripts, chef has to run "Synchronizing Cookbooks" and this step takes 2 to 3 minutes. I understand that chef is downloading dependant cookbooks in this time and checking their hashes.

my question

If I'm using berkshelf and already have the cookbooks saved locally, why does chef spend so long downloading Cookbooks?

Anyone know a way to speed it up?

Upvotes: 4

Views: 1323

Answers (2)

Jim
Jim

Reputation: 23

I ran into a similar situation with my kitchen converge stalling at "Resolving cookbook dependencies with Berkshelf..." After process of elimination, I found that I had several large files in the cookbook path (my VirtualBox .box files actually). I speculated that the converge process was scanning through the files to "chefignore", though I have no evidence to prove it.

I did find this article which alludes to this being a possibility: https://github.com/chef/chef/issues/6323. And it around my version of chef, which is 12.21.31.

So, to speed it up, remove all the non-essential files from the cookbook path. Large sized files or folders with lots of files that add up.

Upvotes: 1

Jonas Linde
Jonas Linde

Reputation: 66

I wondered about this as well and finally did a strace on the chef-client process to see what it was doing. For every cookbook available to the chef client when you run in local mode, it reads through files in all other cookbooks that are available (and to make matters worse it seems to be doing it more than once for every cookbook).

This means the time to synchronize cookbooks in local mode will grow with the square of the number of cookbooks available.

One thing you can do to minimize this time is to remove all unnecessary cookbooks before the run - ie. synchronize your cookbooks manually before hand and remove the ones that will not be used.

As I haven't checked the code, I don't know why it does this, but the real fix would be to change the code to not reread the same files over and over again.

Upvotes: 5

Related Questions