clang 4.0 fails to build clang 3.42 on ubuntu 17.04

Question

I have posted my failures of building llvm 3.42 with gcc6.3 here and here. I am posting separately the failure of clang 4.0 to build llvm 3.42 from source as these are somewhat different developers communities. Here is the script I've used:

svn co https://llvm.org/svn/llvm-project/llvm/tags/RELEASE_342/final llvm
svn co https://llvm.org/svn/llvm-project/cfe/tags/RELEASE_342/final llvm/tools/clang
svn co https://llvm.org/svn/llvm-project/compiler-rt/tags/RELEASE_342/final llvm/projects/compiler-rt
svn co https://llvm.org/svn/llvm-project/libcxx/tags/RELEASE_342/final llvm/projects/libcxx

rm -rf llvm/.svn
rm -rf llvm/tools/clang/.svn
rm -rf llvm/projects/compiler-rt/.svn
rm -rf llvm/projects/libcxx/.svn

cd llvm

CC=clang CXX=clang++ ./configure \
--enable-optimized \
--disable-assertions \
--enable-targets=host \
--with-python="/usr/bin/python2"

make -j `nproc`

And here is the error I got:

[ 17%] Linking CXX executable ../../bin/yaml-bench
../../lib/libLLVMSupport.a(Allocator.cpp.o): In function `llvm::BumpPtrAllocator::Allocate(unsigned long, unsigned long)':
/home/oren/GIT/myLLVMpass/llvm-3.4.2/llvm/lib/Support/Allocator.cpp:(.text+0x377): undefined reference to `__msan_allocated_memory'
/home/oren/GIT/myLLVMpass/llvm-3.4.2/llvm/lib/Support/Allocator.cpp:(.text+0x40b): undefined reference to `__msan_allocated_memory'
/home/oren/GIT/myLLVMpass/llvm-3.4.2/llvm/lib/Support/Allocator.cpp:(.text+0x453): undefined reference to `__msan_allocated_memory'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
utils/not/CMakeFiles/not.dir/build.make:95: recipe for target 'bin/not' failed

EDIT: (after adding @valiano's suggestion) There are still errors, here is the terminal output:

In file included from /home/oren/GIT/LatestKlee/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cc:47:
/usr/include/malloc.h:38:14: error: declaration conflicts with target of using declaration already in scope
extern void *malloc (size_t __size) __THROW __attribute_malloc__ __wur;
             ^
/usr/include/stdlib.h:427:14: note: target of using declaration
extern void *malloc (size_t __size) __THROW __attribute_malloc__ __wur;
             ^
/usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/stdlib.h:65:12: note: using declaration
using std::malloc;
           ^
In file included from /home/oren/GIT/LatestKlee/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cc:47:
/usr/include/malloc.h:41:14: error: declaration conflicts with target of using declaration already in scope
extern void *calloc (size_t __nmemb, size_t __size)
             ^
/usr/include/stdlib.h:429:14: note: target of using declaration
extern void *calloc (size_t __nmemb, size_t __size)
             ^
/usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/stdlib.h:59:12: note: using declaration
using std::calloc;
           ^
In file included from /home/oren/GIT/LatestKlee/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cc:47:
/usr/include/malloc.h:49:14: error: declaration conflicts with target of using declaration already in scope
   ASSEMBLE:  clang_linux/tsan-x86_64/x86_64: /home/oren/GIT/LatestKlee/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_rtl_amd64.S
extern void *realloc (void *__ptr, size_t __size)
             ^
/usr/include/stdlib.h:441:14: note: target of using declaration
extern void *realloc (void *__ptr, size_t __size)
             ^
/usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/stdlib.h:73:12: note: using declaration
using std::realloc;
           ^
In file included from /home/oren/GIT/LatestKlee/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cc:47:
/usr/include/malloc.h:53:13: error: declaration conflicts with target of using declaration already in scope
extern void free (void *__ptr) __THROW;
            ^
/usr/include/stdlib.h:444:13: note: target of using declaration
extern void free (void *__ptr) __THROW;
            ^
/usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/stdlib.h:61:12: note: using declaration
using std::free;
           ^
4 errors generated.

Is there any way out of this? thanks in advance!

Answer 1

That seems to be an issue with LLVM 3.4.2 source code and memory sanitizer headers.

In short, you'll need to patch two source files, lib/Support/Allocator.cpp and lib/Target/X86/X86JITInfo.cpp, for removing usages of the functions__msan_allocated_memory and __msan_unpoison:

sed -i 's/__msan_unpoison/\/\/__msan_unpoison/' ./llvm/lib/Target/X86/X86JITInfo.cpp
sed -i 's/__msan_allocated_memory/\/\/__msan_allocated_memory/' llvm/lib/Support/Allocator.cpp

These function calls are only there for LLVM builds with memory sanitizer instrumentation, and otherwise not needed.

With this patch, you should be able to build LLVM/clang 3.4.2 with clang 4.0 with no further issues.

Reason this happens:

These functions are of the memory sanitizer and they're called from LLVM code assuming that LLVM is built with memory sanitizer instrumentation (-fsanitize=memory), and otherwise, they are disabled (ifdef'd out).

The relevant header code is in include/llvm/Support/Compiler.h:

/// \macro LLVM_MEMORY_SANITIZER_BUILD
/// \brief Whether LLVM itself is built with MemorySanitizer instrumentation.
#if __has_feature(memory_sanitizer)
# define LLVM_MEMORY_SANITIZER_BUILD 1
# include <sanitizer/msan_interface.h>
#else
# define LLVM_MEMORY_SANITIZER_BUILD 0
# define __msan_allocated_memory(p, size)
# define __msan_unpoison(p, size)
#endif

From some reason, the first if clause is chosen, instead of the second one, even though the build is made without memory sanitizer enabled (perhaps the toggling of __has_feature(memory_sanitizer) macro is broken).

See here additional reports regarding this issue.

Edit:

For solving the following issue of tsan failing to build due to malloc redeclarations, you may consider omitting tsan from your LLVM build, as described here.