linux build system tool

Question

Our project got pretty big and our build system does not scale anymore. We are doing cross platform development on linux machines. We have too many platforms to build against and even more build options. We believe that we need to upgrade our Makefile based build environment.

These are the requirements (in an ideal world):

• Fast (so no libtool)
• Can do parallel builds
• Cross compile friendly
• Ccache integration
• Does incremental builds and can short circuit if certain conditions are met (short circuit if a,b,c options have not changed, rebuild if they did)
• Easily scriptable (python integration would be perfect)
• User friendly syntax
• Distributed system. Modules can be developed separately from each other
• Can build third party libraries (that use autotools, cmake ..)
• Can track dependencies between modules (but flexible enough so that modules can be replaced by alternative external ones).
• built-in unit testing support
• Large binaries can be stored separately from the version control and can be downloaded if needed
• Can keep track of open source licenses
• git integration

Are you aware of any tools (or group of tools) that would meet (at least some of) these requirements? Currently I am leaning towards gyp+ninja. But syntax is not very friendly and there is no documentation. So it is a tough sell.

Answer 1

ninja is not supposed to be used by the end-user, rather by some other high level build tool like CMake. And that is really a good option, especially for large and crossplatform projects. It has no built in python support, but you rarely or never need external scripting using CMake - it has tools for most common tasks.

Answer 2

As for C++ is concerned, a very good build sysem is CMake.

Answer 3

You mention Python integration, so SCons sounds like it would fit the bill. It's entirely based around Python (the build scripts are in fact Python scripts), it is very flexible, and it meets quite a number of your other requirements.

From the web site:

SCons is an Open Source software construction tool—that is, a next-generation build tool. Think of SCons as an improved, cross-platform substitute for the classic Make utility with integrated functionality similar to autoconf/automake and compiler caches such as ccache. In short, SCons is an easier, more reliable and faster way to build software.