How exactly does CMake work? Why are so many files generated?

Question

What exactly was going on behind the scenes when for such a small CMakeLists.txt file,

cmake_minimum_required (VERSION 2.6)
project(Tutorial)
add_executable(Tutorial tutorial.cpp)

and such a small tutorial.cpp

int main() { return 0; }

there are so many files generated:

CMakeCache.txt  cmake_install.cmake  Makefile
CMakeLists.txt  tutorial.cpp

And a CMakeFiles folder with so many files and folders:

CMakeCCompiler.cmake               CMakeOutput.log    Makefile.cmake
cmake.check_cache                  CMakeSystem.cmake  progress.marks
CMakeCXXCompiler.cmake             CMakeTmp           TargetDirectories.txt
CMakeDetermineCompilerABI_C.bin    CompilerIdC        Tutorial.dir
CMakeDetermineCompilerABI_CXX.bin  CompilerIdCXX
CMakeDirectoryInformation.cmake    Makefile2

Not understanding what was going on behind the scenes (i.e: why so may files had to be generated and what their purpose was), was the biggest obstacle in being able to learn CMake.

What is the purpose of these files, and when I type cmake ., what exactly is CMake configuring and generating before it builds the project?

Answer 1

Exactly how CMake works is a question for the developers, so this question can't be answered here.

However, we can give a touch of useful guidance as far as when you should use CMake and when you therefore need to worry about how it works. I'm not a fan of "oh, it just works" answers either—because, especially in software, nothing ever "just works" and you always have to get into the nitty-gritty details at some point.

CMake is an industrial-strength tool. It automates several very complex process and takes into account many variables of which you may not be aware, especially as a fairly new developer, probably working with limited knowledge of all the operating systems and build tools CMake can handle. The reason so many files are generated and why things seem so complex is because all of those other systems are complex and must be accounted for and automated.

Additionally, there are the issues of "caching" and other time-saving features of the tool. To understand everything in CMake would mean understanding everything in these build tools and operating systems and all the possible combinations of these variables, which as you can imagine is impossible.

It's important to note that if you're not in charge of managing a large cross-platform build system, and your code base is a few KLOC, maybe up to 100 KLOG, using CMake seems a little bit like using a 100,000 dollar forestry tree removal machine to remove weeds from your 2 feet by 2 feet (60 x 60 cm) flower garden. (By the way, if you've never seen such a machine, you should look for one on YouTube; they're amazing.)

If your build system is small and simple it's likely to be better to just write your own makefiles by hand or script them yourself. When your makefiles become unwieldy or you need to build a version of your system on another platform, then you can switch over to CMake. At that point, you'll have lots of problems to solve and you can ask more focused questions about it. In the meantime, check out some of the great books that have been written about CMake, or even better, write one yourself! 8)

Answer 2

As stated on its website:

CMake is cross-platform, open-source build system for managing the build process of software using a compiler-independent method

In most cases, it is used to generate project/make files. In your example, it has produced Makefile which are used to build your software (mostly on Linux/Unix platform).

CMake allows to provide cross-platform build files that would generate platform-specific project/make files for a particular compilation/platform.

For instance, you may to try to compile your software on Windows with Visual Studio, and then with proper syntax in your CMakeLists.txt file you can launch

cmake .

inside your project's directory on the Windows platform. CMake will generate all the necessary project/solution files (.sln, etc.).

If you would like to build your software on the Linux/Unix platform, you would simply go to source directory where you have your CMakeLists.txt file and trigger the same cmake . and it will generate all files necessary for you to build software via a simple make or make all.

Here you have some very good presentation about key CMake functionalities: Learning CMake (Pau Garcia i Quiles)

If you'd like to make platform-dependent library includes / variable definitions, etc., you can use this syntax in the CMakeLists.txt: file:

IF(WIN32)
   ...do something...
 ELSE(WIN32)
   ...do something else...
 ENDIF(WIN32)

There are also a lot of commands with use of which you are able to prevent the build from failing and in place CMake will notify you that, for instance, you do not have Boost libraries filesystem and regex installed on your system. To do that you can use the following syntax:

find_package(Boost 1.45.0 COMPONENTS filesystem regex)

Having checked that it will generate the makefiles for appropriate system/IDE/compiler.

Answer 3

The secret is that you don't have to understand what the generated files do.

CMake introduces a lot of complexity into the build system, most of which only pays off if you use it for building complex software projects.

The good news is that CMake does a good job of keeping a lot of this messiness away from you: Use out-of-source builds and you don't even have to look at the generated files. If you didn't do this so far (which I guess is the case, since you wrote cmake .), please check them out before proceeding. Mixing the build and source directory is really painful with CMake and is not how the system is supposed to be used.

In a nutshell: Instead of

cd <source_dir>
cmake .

always use

cd <build_dir_different_from_source_dir>
cmake <source_dir>

I usually use an empty subfolder build inside my source directory as build directory.

To ease your pain, let me give a quick overview of the relevant files which CMake generates:

• Project files/Makefiles - What you are actually interested in: The files required to build your project under the selected generator. This can be anything from a Unix Makefile to a Visual Studio solution.
• CMakeCache.txt - This is a persistent key/value string storage which is used to cache value between runs. Values stored in here can be paths to library dependencies or whether an optional component is to be built at all. The list of variables is mostly identical to the one you see when running ccmake or cmake-gui. This can be useful to look at from time to time, but I would recommend to use the aforementioned tools for changing any of the values if possible.
• Generated files - This can be anything from autogenerated source files to export macros that help you re-integrate your built project with other CMake projects. Most of these are only generated on demand and will not appear in a simple project such as the one from your question.
• Anything else is pretty much noise to keep the build system happy. In particular, I never needed to care about anything that is going on inside the CMakeFiles subdirectory.

In general you should not mess with any of the files that CMake generates for you. All problems can be solved from within CMakeLists.txt in one way or the other. As long as the result builds your project as expected, you are probably fine. Do not worry too much about the gory details - as this is what CMake was trying to spare you of in the first place.