CODEWITHSUNDEEP
tensorflowbazelsyntaxnet
Rushat Rai
Rushat Rai

Reputation: 793

How do I specify the number of CPU cores bazel uses?

I am building syntaxnet (tensorflow fork) with bazel. It's working very slowly and keeps hanging up.

The last time I had this problem (with caffe) someone told me to change the number of cores I was using by adding -j4. In bazel this command did not work. Any custom command for bazel like this?

CPU Specs: 3.8ghz clock, Quadcore

CPU Model: AMD 4800(or something along those lines).

Upvotes: 2

Views: 7652

Answers (2)

dmansfield
dmansfield

Reputation: 1128

I added --jobs 4 and it managed to get through. It's probably the same as -j4.

Complete command line was:

bazel test --jobs 4 --genrule_strategy=standalone syntaxnet/... util/utf8/...

Upvotes: 4

sage
sage

Reputation: 5514

As Guy Coder describes in an answer to a similar question, try --local_resources. In my limited experience, it seems that Bazel consumes approximately double the number of cores I set and sometimes consumes all cores for many seconds. Even with bazel test -c opt --local_resources 20000,1,0.25 I see significant load on my system. (Following is his answer)

From Bazel User Manual

--local_resources availableRAM,availableCPU,availableIO

This option, which takes three comma-separated floating point arguments, specifies the amount of local resources that Bazel can take into consideration when scheduling build and test activities. Option expects amount of available RAM (in MB), number of CPU cores (with 1.0 representing single full core) and workstation I/O capability (with 1.0 representing average workstation). By default Bazel will estimate amount of RAM and number of CPU cores directly from system configuration and will assume 1.0 I/O resource.

If this option is used, Bazel will ignore both --ram_utilization_factor.

Upvotes: 0

Related Questions