CODEWITHSUNDEEP
makefilebuildcmakeparallel-builds
Roman Dmitrienko
Roman Dmitrienko

Reputation: 4205

cmake and parallel building with "make -jN"

I'm trying to setup a parallel CMake-based build for my source tree, but when I issue

$ cmake .
$ make -j2

I get:

jobserver unavailable: using -j1.  Add '+' to parent make rule

as a warning. Does anyone have an idea if it is possible to fix it somehow?

Upvotes: 19

Views: 14376

Answers (3)

scrutari
scrutari

Reputation: 1618

As pointed out by @Carlo Wood in his comment to this answer, trying to convince cmake to add + to the beginning of the command in the cmake-generated makefile is not possible.

A work-around I found is to shield underlying make command from the make flags coming from cmake. This can be done by setting environment variable MAKEFLAGS to empty string for the custom command:

COMMAND ${CMAKE_COMMAND} -E env
        MAKEFLAGS=
        make <your target and make options>

Hope this helps.

Upvotes: 0

ulidtko
ulidtko

Reputation: 15560

In my case (with CMake 3.5.2) the trivial cd build && cmake .. && make -j5 works just fine.

But, I do get the jobserver unavailable error when building custom targets (as dependencies of other targets) via the cmake --build . --target foo idiom.

Like this:

add_custom_target(buildroot
   COMMAND ${CMAKE_COMMAND} --build . --target install
   COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc

— so that the user can make deb and it Just Works. CMake will regenerate makefiles if needed, run the compilation, install everything exactly as with make install, and then run my custom scripts to package up the populated buildroot into whatever shape or form I need.

Sure enough, I'd like to make -j15 deb — but that fails.

Now, as explained on the mailing list by CMake devs, the root cause lies, surprisingly (or not), within GNU Make; there is a workaround.

The root cause is that make will not pass its jobserver environment to child processes it thinks aren't make.

To illustrate, here's a process tree (ps -A f) branch: … \_ bash \_ make -j15 deb \_ make -f CMakeFiles/Makefile2 deb \_ make -f CMakeFiles/buildroot.dir/build.make CMakeFiles/buildroot.dir/build \_ /usr/bin/cmake --build . --target install ⦿ \_ /usr/bin/gmake install …

At ⦿ point, make drops jobserver environment, ultimately causing single-threaded compilation.

The workaround which worked great for me, as given away in the linked email, is to prefix all custom commands with +env. Like this:

add_custom_target(buildroot
   #-- this ↓↓↓ here -- https://stackoverflow.com/a/41268443/531179
   COMMAND +env ${CMAKE_COMMAND} --build . --target install
   COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc

In the end, this appears in the rule for buildroot in the appropriate makefile (CMake generates a bunch of them), and causes GNU Make to behave properly and respect -j.

Hope this helps.

Upvotes: 9

apenwarr
apenwarr

Reputation: 11036

In the generated Makefile, when calling into a sub-make it needs to either use $(MAKE) (not just 'make') or else precede the line with a +. That is, a rule should look like this:

mysubdir:
    $(MAKE) -C mysubdir

or like this:

mysubdir:
    +make -C mysubdir

If you don't do it one of those two ways, make will give you that warning.

I don't know anything about cmake, so maybe it's generating Makefiles that aren't correct. Or maybe you did something incorrectly on your end.

Upvotes: 25

Related Questions