Compile boost as universal library (Intel and Apple Silicon architectures)

Question

I am trying to build boost library as dylib on MacOS. I need to build it for both the Intel architecture and the upcoming Apple Silicon (arm64) architecture.

I downloaded boost and ran the following commands:

./bootstrap.sh
./b2 -address-model=64 architecture=combined -a

lipo -archs always shows produced dylibs architecture is x86_64.

I have Xcode12 beta and MacOS Catalina 10.15.7,

I can build a sample universal library if I create a project in Xcode and set archs arm64 x86_64 in build settings.

Running command ./b2 cxxflags="-arch arm64 -arch x86_64" fails with following errors:

    "clang++" -x c++ -fvisibility-inlines-hidden -m64 -O3 -Wall -fvisibility=hidden -Wno-inline -arch arm64 -arch x86_64 -ftemplate-depth-255 -fvisibility=hidden -fvisibility-inlines-hidden -DBOOST_ALL_NO_LIB=1 -DNDEBUG -I"." -c -o "bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o" "libs/serialization/src/polymorphic_xml_iarchive.cpp"

...failed clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o...
clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_oarchive.o
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:32:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_dev_t;         /* dev_t */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
        ^

Answer 1

I needed this with CMake. while ABI was a little confusing. Based on Boost/CMake docs 1, I was able to get a Universal (x86_64 arm64) on macOS/Xcode by defining the following:

BOOST_CONTEXT_ABI=sysv BOOST_CONTEXT_ARCHITECTURE=combined

Internally 2 it seems to include the AAPCS for arm64 and the sysv for x86_64.

Answer 2

Checked for macOS M1 building boost 1.82.0, successfully build fat binaries with both architecture included:

./bootstrap.sh
./b2 -a address-model=64 architecture=arm+x86

To install systemwide:

sudo ./b2 -a address-model=64 architecture=arm+x86 install

Checked that it linked successfully. Feel free to make sure with:

lipo -info /usr/local/lib/libboost_chrono.dylib
Architectures in the fat file: /usr/local/lib/libboost_chrono.dylib are: x86_64 arm64

Hope it helps.

Answer 3

After spending considerable amount of time here:

https://github.com/bfgroup/b2/issues/105

I was able to compile boost on arm64. The command line is:

./b2 architecture=arm address-model=64 asmflags=--target=arm64-apple-darwin21.2.0 cflags=--target=arm64-apple-darwin21.2.0 cxxflags=--target=arm64-apple-darwin21.2.0 linkflags=--target=arm64-apple-darwin21.2.0 -s NO_LZMA=1 -s NO_ZSTD=1 abi=aapcs

the LZMA and ZSTD flags are there since I did not have an universal binary for those libs on my machine.

Answer 4

I tried the script on boost 1.78, but the limo stage fails: lipo: specifed architecture type (arm64) for file (arm64/libboost_atomic.dylib) does not match its cputype (16777223) and cpusubtype (3) (should be cputype (16777228) and cpusubtype (0))

The seems wrong indeed: #file arm64/libboost_wserialization.dylib arm64/libboost_wserialization.dylib: Mach-O 64-bit dynamically linked shared library x86_64

Clues ?

Answer 5

I'm on Monterey with an M1 (XCode 13.1) and failed to get any of the other answers to work, but I combined stuff from Navan and Petr along with some other bits to get the libboost_coroutine to work on x86_64. I arrived at the following script which builds all the boost libraries (why not, it doesn't take long on the M1):

#!/bin/sh

rm -rf arm64 x86_64 universal stage bin.v2
rm -f b2 project-config*
./bootstrap.sh cxxflags="-arch x86_64 -arch arm64" cflags="-arch x86_64 -arch arm64" linkflags="-arch x86_64 -arch arm64"
./b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs cxxflags="-arch arm64" cflags="-arch arm64" linkflags="-arch arm64" -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang-darwin target-os=darwin architecture=x86 cxxflags="-arch x86_64" cflags="-arch x86_64" linkflags="-arch x86_64" abi=sysv binary-format=mach-o -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do 
  lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib); 
done
for dylib in universal/*; do
  lipo $dylib -info;
done

This script prints out the lipo info for each dylib in the universal directory, so you can see quickly that every one has both x86_64 and arm64 inside. I had the following result, maybe it's useful for you to see if your output matches:

Architectures in the fat file: universal/libboost_atomic.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_chrono.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_container.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_context.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_contract.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_coroutine.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_date_time.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_filesystem.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_graph.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_iostreams.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_locale.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_log.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_log_setup.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_numpy27.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_prg_exec_monitor.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_program_options.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_python27.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_random.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_regex.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_serialization.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_addr2line.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_basic.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_noop.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_system.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_thread.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_timer.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_type_erasure.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_unit_test_framework.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_wave.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_wserialization.dylib are: x86_64 arm64 

Answer 6

I followed the answer by Navan Chauhan which seems to be the way to go on macOS 10.15.7 with Xcode 12.3.

My only issue was that the build system was adding additional -arch armv4t clang option (or something similar) which fails the build. Using the architecture=combine cannot work as well as it is able to manage Intel and PowerPC only. What does work for me instead is:

1. bootstrap.sh --with-toolset=clang-darwin and cxxflags, cflags and linkflags set to -arch x86_64 -arch arm64 plus additional options.
2. x86_64 build: b2 toolset=clang-darwin target-os=darwin architecture=x86 stage and cxxflags, cflags and linkflags set to -arch x86_64 only (plus additional options).
3. arm64 build: b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs stage and cxxflags, cflags and linkflags set to -arch arm64 only (plus additional options).
4. Merge libraries with lipo…

I've been forced to set the ABI also as the automatic-guess-something is not able to recognize Apple Silicon goal therefor some assembly stuff is not compiled – hence missing symbols later on during the build. Finally the armv4t issue is solved by the clang-darwin toolset.

The only caveat of this at the moment is that both builds go into the same architectural/configuration directories which can be solved by custom user-config.jam or by immediate call to install just after each stage which is what I did.

Works on:

• Host platform: Intel
• macOS 10.15.7
• Xcode 12.3 (with Command Line Tools installed)
• Boost 1.75.0 (dependent compression and ICU libraries already compiled as universal)

Answer 7

The following works for me on Big Sur using Xcode 12.2. On Catalina 10.15.7 (19H15) I get the same error messages as OP.

./bootstrap.sh --with-libraries=regex,date_time 
./b2 architecture=combined cxxflags="-arch x86_64 -arch arm64"

Answer 8

You don't need to use lipo, you can pass -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk to cxxflags and it will build a universal library out of the box.

Answer 9

I was facing the same issue and came across this answer, which was for creating universal binaries for i386 and x86_64.

To summarise the answer, you need to run ./b2 twice ( with different toolsets as clang always builds for x86_64, even if you pass the appropriate CXXFlags

This is the script I used to first individually generate the libraries I required and then combined them using lipo

#!/bin/sh

rm -rf arm64 x86_64 universal
./bootstrap.sh --with-toolset=clang --with-libraries=thread,system,filesystem,program_options,serialization 
./b2  cxxflags="-arch arm64" toolset=darwin -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang cxxflags="-arch arm64 -arch x86_64" -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do 
  lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib); 
done