CODEWITHSUNDEEP
c++boostcentosboost-filesystemappimage
Marek R
Marek R

Reputation: 38112

Setting custom locale on boost::filesystem throws an exception on CentOS

Revised version:

Main problem

I have problem to properly configure AppImage, so locale with UTF-8 encoding could be applied for boost::filesystem::path::imbue. Code works on any platform when build on this specific platform. Problem appears when wrapping application build on Ubuntu 24.04 with clang-18 with AppImage. This AppImage works on most distributions and fails on CentOS 7.9.

Cpp code

#include <iostream>
#include <boost/program_options/detail/utf8_codecvt_facet.hpp>
#include <boost/filesystem.hpp>

void setupGlobalCppLocale()
try
{
    auto utf8Locale =  std::locale{std::locale::classic(), new boost::program_options::detail::utf8_codecvt_facet{}};
    std::locale::global(utf8Locale);
    std::cout << "Globa locale set succesfully\n";
    std::cout << "Setup boost::filesystem::path::imbue..." << std::endl;
    boost::filesystem::path::imbue(utf8Locale);
    std::cout << "localeboost::filesystem::path::imbue set succefully" << std::endl;
}
catch (const std::exception& e)
{
    std::cerr << "Failed to configure global locale: " << e.what() << std::endl;
}

int main(int argc, const char* argv[])
{
    setupGlobalCppLocale();
}

When build on CentOS like this it works:

$ g++ -std=c++11 test.cpp -lboost_{system,filesystem,program_options} -o Manual -O2
$ ./Manual 
Globa locale set succesfully
Setup boost::filesystem::path::imbue...
localeboost::filesystem::path::imbue set succefully
$

AppImage

Here is AppImageBuilder.yml

# appimage-builder recipe see https://appimage-builder.readthedocs.io for details
version: 1
AppDir:
  path:
    /home/marekr22/localeTest/AppDir
  app_info:
    id: com.locale_experiment.app
    name: experiment_locale
    version: 1.0.0
    exec: experiment_locale
    exec_args: $@
  apt:
    arch:
    - amd64
    allow_unauthenticated: true
    sources:
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble main restricted
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble-updates main restricted
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble universe
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble-updates universe
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble multiverse
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble-updates multiverse
    - sourceline: deb http://pl.archive.ubuntu.com/ubuntu/ noble-backports main restricted universe multiverse
    - sourceline: deb http://security.ubuntu.com/ubuntu noble-security main restricted
    - sourceline: deb http://security.ubuntu.com/ubuntu noble-security universe
    - sourceline: deb http://security.ubuntu.com/ubuntu noble-security multiverse
      key_url: 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x871920d1991bc93c'
    include:
    - libc6
    - libstdc++6
    - locales
    # - lib64gcc-s1
  files:
    include: 
    - /usr/lib/locale/
    exclude:
    - usr/share/man
    - usr/share/doc/*/README.*
    - usr/share/doc/*/changelog.*
    - usr/share/doc/*/NEWS.*
    - usr/share/doc/*/TODO.*
  runtime:
    env:
      GCONV_PATH: $APPDIR/runtime/compat/usr/lib/x86_64-linux-gnu/gconv
      APPDIR_LIBRARY_PATH: $APPDIR/lib/x86_64-linux-gnu:$APPDIR/usr/lib/x86_64-linux-gnu:$APPDIR/usr/lib/x86_64-linux-gnu/gconv
      # LOCPATH: $APPDIR/usr/lib/locale
      LC_ALL: en_US.UTF-8
      LANG: en_US.UTF-8
  after_runtime: |
    COMPAT_RUNTIME=$TARGET_APPDIR/runtime/compat
    if [ -e $COMPAT_RUNTIME/usr ] && [ -d $COMPAT_RUNTIME/usr/lib64 ]; then
      ln -s --relative $COMPAT_RUNTIME/usr/lib64 $COMPAT_RUNTIME
    fi
AppImage:
  arch: x86_64
  update-information: guess

This is build more or less like this:

clang++ -std=c++20 test.cpp -I<PATH TO STATIC BUILD OF BOOST INCLUDE> -L<PATH TO STATIC BUILD OF BOOST LIB> -lboost_{system,filesystem,program_options} -o experiment_locale -O2
mkdir AppDir
cp experiment_locale AppDir/
appimage-builder --recipe AppImageBuilder.yml

It suppose to work without problems on multiple Linux distributions: RedHat Enterprise Linux (RHEL) 7.9+, Centos 7.9+, Ubuntu 18.04+, Debian 10+, SUSE Linux Enterprise Server (SLES) 15, Oracle Linux 8, Amazon Linux 2, Amazon Linux 2023, Alma Linux 8+, Rocky Linux 8+, Linux Mint 20+. And I do not have any (almost) complains.

Problem appears only on Centos 7.9+ (I reproduce issue in my vm). Exception is thrown, in logs I can see:

$ strace -o strace_logs_centos.txt ./experiment_locale-1.0.0-x86_64.AppImage 
Globa locale set succesfully
Setup boost::filesystem::path::imbue...
Failed to configure global locale: locale::facet::_S_create_c_locale name not valid

As you can see custom locale is accepted by std::locale::global(utf8Locale);, but error is reported in case of boost::filesystem::path::imbue(utf8Locale);

So problem is not custom locale itself, but the way boost tries to utilize it.

So why it is failing? And how I can fix it?

When trying investigating that I used strace.
Here are strace logs when (sorry this is in form of links, but SO question size limit was reached):

I'm unable to find in logs clue for a solution. I did tried multiple changes to AppImage definition without luck. Outcome is always same (or worse - I even got same result on Ubuntu).

Upvotes: 3

Views: 61

Answers (1)

sehe
sehe

Reputation: 393664

Reproducer: docker run --rm -i -t centos:7 and then:

sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo
sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo
sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo

sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*

yum install -y git gcc-c++ make cmake boost boost-devel
cat > test.cpp <<CPP
#include <boost/filesystem.hpp>
#include <boost/program_options.hpp>
#include <boost/program_options/detail/utf8_codecvt_facet.hpp>
#include <iostream>
#include <locale>

int main() try {
    std::locale defLocale{};

    auto customUtf8Locale = std::locale( //
        defLocale                        //
        , new boost::program_options::detail::utf8_codecvt_facet{} //
    );
    std::locale::global(customUtf8Locale);

    std::cout << "Setting boost::filesystem::path::imbue locale..." << std::endl;
    boost::filesystem::path::imbue(customUtf8Locale);

} catch (std::exception const& e) {
    std::cerr << "Exception " << e.what() << std::endl;
    return 1;
}
CPP

Now building and running works fine:

g++ -std=c++11 test.cpp -lboost_{system,filesystem,program_options}

enter image description here

You're welcome to add information to the question to highlight what you're doing differently.

UPDATE: Static Runtime and Link

Downloading Boost 1.86 and building it manually for static linking:

yum install wget bzip2 python2-devel libstdc++-static glibc-static
wget https://archives.boost.io/release/1.86.0/source/boost_1_86_0.tar.bz2
tar xf boost_1_86_0.tar.bz2 
cd boost_1_86_0
./bootstrap.sh 
./b2 runtime-link=static cxxstd=11 --with-{system,filesystem,program_options}

Now again/still compiling and running works fine:

cd ~
g++ -static -std=c++11 test.cpp -I /root/custom/boost_1_86_0 /root/custom/boost_1_86_0/stage/lib/libboost_{system,filesystem,program_options}.a
./a.out

Upvotes: 3

Related Questions