C++ internal compiler error

Question

Since many weeks our compilation server is crashing randomly while compiling our C++ code. Sometimes the compilation failed and we have the following error :

/usr/include/c++/7/future:429:7: internal compiler error: Segmentation fault

The error is always raised from system libraries (but not always the same) and at different step of the compilation process.

We have tried to increase the size of RAM up to 10 GB and the size of the swap (up to 5GB) but the issue has not been solved. We have also tried multiple version of the cc compiler but without success.

We have a set a machine but the issue is only reproducible on out compilation server. We have to fix it because this server is part of our continuous integration chain.

The source code is composed of about 10000-20000 line of codes (not really much) but we use some template.

Does someone knows how to solve or investigate this error ?

System information:

compiler = c++ 
compiler version = c++ (Ubuntu 7.2.0-1ubuntu1~16.04) 7.2.0
compilation tools = cmake and make
ubuntu-xenial
RAM  = 10G
Swap = 5G
NbCPU = 4

Thank you very much for your help

Answer 1

So you've got intermittent errors in (presumably well tested) compiler internals from (presumably also well tested) system libraries, and the problems are reproducible on multiple compiler versions, but only on this single machine. That points towards a hardware issue.

Bad RAM seems like a good candidate. A C++ compiler processing a moderately sized code base is likely to crash from e.g. random bit flips at least some of the time.

You should test your RAM (or just swap it out and see if the failures go away).