CODEWITHSUNDEEP
c++linuxgcccompilationmultiprocessing
Jay D
Jay D

Reputation: 3307

how to optimize build in linux

How to expedite/optimize compilation of a huge project on Linux on a multi processor machine so that compilation is done in parallel utilizing all processors.

Any suggestions regarding special flags , tweaks would be helpful.

Regards,-J

Upvotes: 0

Views: 272

Answers (5)

Michael S.
Michael S.

Reputation: 51

You can use distcc tool to perform compilation of different source files on different Linux machines from a predefined machines-pool. Obviously "-j" in the make command line must be used. Additionally you should try to avoid recursive Makefiles, i.e. invoking "make -C $sub_dir" recursive from Makefile - this slows make process on huge projects.

Upvotes: 0

Non-maskable Interrupt
Non-maskable Interrupt

Reputation: 3911

You should setup dependency rule and tell the compiler to emit them(-MD -MP for gcc), for example:

YOUR_DEP    :=$(patsubst %.o,%.d,$(YOUR_OBJ))
-include $(YOUR_DEP)
...

%_c.o: %.c Makefile
@echo [GCC ] $<
@$(CC) $(CFLAGS) -MD -MP $< -o $@
%_cpp.o: %.cpp Makefile
@echo [CXX ] $<
@$(CXX) $(CXXFLAGS) -MD -MP $< -o $@

This way make can do the dependency checking. However, parallel make -j sometime cause trouble for huge project with multiple level of dependency.

Upvotes: 1

Employed Russian
Employed Russian

Reputation: 213596

If you really have a huge project, say several million lines of C++, and a couple thousand CPUs to throw at it, you might want to look at how Google solves its build problem.

Upvotes: 2

cvoinescu
cvoinescu

Reputation: 856

It depends on the build system you use. If it's GNU make, simply adding -j (or --jobs) should parallelize the build. It may or may not work on a huge project, depending on how well-behaved the makefile is.

See also the manual.

Upvotes: 1

Fred Foo
Fred Foo

Reputation: 363597

make -j <NUM_CPUS>

(See make(1).)

Upvotes: 1

Related Questions