CODEWITHSUNDEEP
rubyperformanceparallel-processingbundler
mahemoff
mahemoff

Reputation: 46379

Bundle: How many parallel jobs?

Bundle supports parallel processing with --jobs:

The --jobs option (-j for short) installs gems in parallel. For example, bundle install -j4 will use 4 workers. We've seen speedups of 40-60% on fresh bundle installs. To always install in parallel, run bundle config --global jobs 4 or set BUNDLE_JOBS.

4 is the example used here, but why not 2 or 8? Is there an optimum number or a way to calculate the optimum number of parallel processes? If the time is bound by network activity, I'd have thought it should be high, e.g. 64, or if bound by computation, then maybe the number of cores on the machine.

Upvotes: 7

Views: 4117

Answers (2)

Pere Joan Martorell
Pere Joan Martorell

Reputation: 3103

getconf works both in Mac OS X and Linux, just in case you need it to be compatible with both systems:

$ getconf _NPROCESSORS_ONLN
12

Some people suggest to use the total number of cores minus 1, but from my experience you can use all of them without issues:

bundle install --jobs `getconf _NPROCESSORS_ONLN`

Upvotes: 5

Mr. S
Mr. S

Reputation: 104

It all depends on how many cores your CPU has and the available memory. Assuming you have decent memory amount, you can use the number of cores as your -j parameter. If you are on linux, you can run this command to list your cores:

cat /proc/cpuinfo | grep processor

Upvotes: 3

Related Questions