Error while using a newer version of glibc

Question

I am trying to install tensorflow on a linux server where I am just a user without the root permission. And I cannot transfer files to/from it as I ssh to it through a jump server. The system is as following :

Linux THENAME_OF_SURVER 2.6.32-573.18.1.el6.x86_64 #1 SMP Tue Feb 9 22:46:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

I installed the tensorflow through pip install tensorflow and a tensorflow program would display the following:

ImportError: /lib64/libc.so.6: version `GLIBC_2.16' not found

I installed a new version of glibc

git clone git://sourceware.org/git/glibc.git cd glibc git checkout --track -b local_glibc-2.16 origin/release/2.16/master mkdir build cd build ../configure --prefix=/home/MYNAME/dependency/glibc-2.16 make -j4 make install

Followed the instructions online, I changed the environment variables through:

export LD_LIBRARY_PATH=/home/MYNAME/dependency/glibc-2.16/lib

BUT this leads me to a problem: I cannot use any command. For example, I called ls and it would warn me like this:

ls: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

I then followed another instruction to run the command as following:

/home/MYNAME/dependency/glibc-2.16/lib/ld-linux-x86-64.so.2 --library-path /home/MYNAME/dependency/glibc-2.16/lib:$LD_LIBRARY_PATH:/path/to/gcc-5.2.0/lib64:/usr/lib64/:/usr/lib64/ ls (I do not know where to find the similar folder as gcc-5.2.0, my which gcc shows /usr/local/sbin/gcc, but it links to /usr/local/gcc-5.3.0/bin/gcc, which doesn't have a lib64 subfolder)

But then it came with the following warning:

ls: error while loading shared libraries: ls: cannot open shared object file

I know that I can use ls again by export the variable to empty. But I still cannot use the new version of glibc. Could anyone help me with how to correctly link the new glibc? Any suggestions would be appreciated!

EDIT: So the progress is as following:

1. LD_LIBRARY_PATH=/home/MYNAME/dependency/glibc-2.16/lib python
   would result in python: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

2. /home/MYNAME/dependency/glibc-2.16/lib/ld-2.16.so python
   would result in python: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

EDIT2 & SUMMARY:

To make Employed Russian's answer more detailed, I would paste my final solutions here.

My goal is to use tensorflow in Python on a server that I do not have the root permission. I was warned that ImportError: /lib64/libc.so.6: version 'GLIBC_2.16' not found when import tensorflow.

Based on Employed Russian's answer, I used the following to run my command:

LD_LIBRARY_PATH=/home/USERNAME/dependency/glibc-2.17/lib/:/lib64/:/usr/local/gcc-5.3.0/lib64/ /home/USERNAME/dependency/glibc-2.17/lib/ld-2.17.so /home/USERNAME/anaconda2/bin/python

Split the command into the following parts (I would use ??? to represent the paths that are different for different people.):

1. LD_LIBRARY_PATH=
   • this part deals with dependencies
   • : means split
   • ???/glibc-2.17/lib/
   • /lib64/ and /usr/local/gcc-5.3.0/lib64/: I found these folders by find / -name 'libgcc_s.so.1' because I was
2. /???/glibc-2.17/lib/ld-2.17.so
3. /???/python the path of your executable. For Python, import sys; print(sys.executable) to see your Python path.

Other things:

1. glibc-2.17 is download from gnu. I chose 2.17 because tensorflow needs 2.17 and 2.17 works fine.
2. There is another problem of this solution. I sometimes need to call shell command in Python like os.system('ls') or os.system('python xxx.py'). But it warned me as following if I used it in its normal way: sh: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument and I haven't found a good enough solution for this.

Answer 1

I found the information provided by the Employed Russian above helpful, but what he wrote didn't actually fix my issue. That's because the binary I as using was statically. It was the CLI tools this binary called that presented my real problem. And like others have said, using environment variables only made the problem worse, because it just left me with an unusable shell.

In my case I wanted to get the latest Vagrant version (2.2.19) running on a CentOS 6.10 server. I was hoping it would make an occasional issue I ran into disappear. But the RPM for Vagrant doesn’t install a single binary. Rather it sets up an embedded directory, with all of the CLI tools, and the libraries used by those the CLI tools available beneath. And of course they all required various version of GNU libc that were newer than what I had installed.

The fix for me at least meant compiling, and then installing the required GNU libc version to into it’s own path. And then patching all of the dynamically linked binaries/libraries installed by Vagrant to use the newer GNU libc. On top of that, I also had to had to make sure the various links to libraries within the embedded directory were preserved (see below for more on this). This is the script I wroute to make problems go away (at least, hopefully, for little while):

#!/bin/bash

cd $HOME/
GLIBC_VERSION="2.17"
GLIBC_PREFIX="/usr/glibc/"
VAGRANT_VERSION="2.2.19"

# Install the basic build system utilities.
yum groupinstall -y "Development tools"
yum install -y curl patchelf

# Grab the tarball with the GNU libc source code.
curl -Lfo glibc-${GLIBC_VERSION}.tar.gz "https://ftp.gnu.org/gnu/glibc/glibc-${GLIBC_VERSION}.tar.gz"
echo "a3b2086d5414e602b4b3d5a8792213feb3be664ffc1efe783a829818d3fca37a  glibc-${GLIBC_VERSION}.tar.gz" | sha256sum -c || exit 1

# Extract the secrets and get ready to rumble.
tar xzvf glibc-${GLIBC_VERSION}.tar.gz

# The configure script requrires an independent build directory.
mkdir -p glibc-build && cd glibc-build

# Configure glibc with a GLIBC_PREFIX so it doesn't conflict with distro libc files..
../glibc-${GLIBC_VERSION}/configure --prefix="${GLIBC_PREFIX}" --libdir="${GLIBC_PREFIX}/lib" \
--libexecdir="${GLIBC_PREFIX}/lib" --enable-multi-arch

# Compile and then install GNU libc.
make -j8 && make install

# Download and install Vagrant.
curl -Lfo vagrant_${VAGRANT_VERSION}_x86_64.rpm "https://releases.hashicorp.com/vagrant/${VAGRANT_VERSION}/vagrant_${VAGRANT_VERSION}_x86_64.rpm"
echo "990e8d2159032915f21c0f1ccdcbca1a394f7937e06e43dc1dabe605d208dc20  vagrant_${VAGRANT_VERSION}_x86_64.rpm" | sha256sum -c || exit 1
yum install -y vagrant_${VAGRANT_VERSION}_x86_64.rpm

# Patch the binaries and shared libraries inside the Vagrant directory, so they use the new version of GNU libc.
(find /opt/vagrant/ -type f -exec file {} \; )| grep "dynamically linked" | awk -F':' '{print $1}' | while read FILE ; do
  patchelf --set-rpath /opt/vagrant/embedded/lib:/opt/vagrant/embedded/lib64:/usr/glibc/lib:/usr/lib64:/lib64:/lib --set-interpreter /usr/glibc/lib/ld-linux-x86-64.so.2 "${FILE}"
done

The script should be pretty easy to understand, and adapt to whatever MacGuffin you want to make work. The only tricky part was the rpath I passed to patchelf. I had to make sure the it also included the library paths beneath embedded, or the dependency references internal to the Vagrant get broken.

P.S. Don't forget the update the hashes for any file you down. In particular, you need to compile/install a different version of GNU libc, you will need to update that hash to match the version you want to use.

Answer 2

In my case it was centos 6 with python for pytorch.

I had errors like, etc.:

libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/evaldsu/.conda/envs/conda_env/lib/python3.6/site-

I installed alongside glibc-2.17 in local dir /opt/exp_soft/tools

then I installed in conda env patching tool (can install using other tools as well):

conda install -c conda-forge patchelf

then I patched binary of python to use different glibc path (you can do this with any binary). Be aware that it will change you python binary.

patchelf --set-rpath /opt/exp_soft/tools/glibc-2.17/lib:$HOME/.conda/envs/conda_inf/lib:/usr/lib64:/lib64:/lib --set-interpreter /opt/exp_soft/tools/glibc-2.17/lib/ld-linux-x86-64.so.2 /home/evaldsu/.conda/envs/conda_inf/bin/python3.6

Another option is just install this script if you have full admin access:

https://gist.github.com/harv/f86690fcad94f655906ee9e37c85b174

Answer 3

export LD_LIBRARY_PATH=/home/MYNAME/dependency/glibc-2.16/lib

This answer explains why LD_LIBRARY_PATH doesn't work, and what you should do instead.

I read your post and tried ...
python: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

The error usually means that you have a mismatch between ld-linux and libc.so.6. They must match.

If you are using direct loader invocation via /home/MYNAME/.../ld-2.16.so, you must also arrange for /home/MYNAME/.../libc.so.6 to be loaded.

You can do that by passing --library-path ... to ld-2.16.so, or setting LD_LIBRARY_PATH appropriately.

Your command with ld-2.16 --library-path ... ls is almost correct. The thing you are missing is that ld-2.16 will not search your PATH. You need to give it full pathname: ld-2.16 --library-path ... /bin/ls.