CODEWITHSUNDEEP
cperlldlibxml2dynamic-loading
x-yuri
x-yuri

Reputation: 18973

LD_LIBRARY_PATH doesn't take effect for LibXML.so

XML::LibXML is a Perl library. LibXML.so is a part of the library. I'm trying to make XML::LibXML use custom built libxml2. But providing LD_LIBRARY_PATH doesn't make any difference:

$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f66af5e9000)

$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f2d26ae3000)

$ ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f878cbc8000)

$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
    libxml2.so.2 => /home/yuri/_/libxml2/.libs/libxml2.so.2 (0x00007f6f8f5d8000)

What am I doing wrong? Is there a way to handle it? The goal is to make Perl script use custom built libxml2 (to investigate some issue).

UPD

$ find /usr/lib -name libxml2.so.2
/usr/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2/libxml2.so.2

$ find /usr/local/lib -name libxml2.so.2

Upvotes: 2

Views: 1845

Answers (2)

x-yuri
x-yuri

Reputation: 18973

tl;dr

ExtUtils::MakeMaker hardcodes paths of the shared libraries in the binaries by means of RPATH entry in .dynamic section. But you can preload the needed library,

$ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so

or make the loader ignore RPATH entry,

$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs /usr/lib/ld-linux-x86-64.so.2 --inhibit-rpath /abs/path/to/LibXML.so /abs/path/to/perl ./1.pl

or convert RPATH entry into RUNPATH one,

$ chrpath --convert blib/arch/auto/XML/LibXML/LibXML.so

or delete RPATH entry,

$ chrpath --delete blib/arch/auto/XML/LibXML/LibXML.so

or specify XMLPREFIX variable,

$ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build

Under the hood

ExtUtils::Liblist::ext() takes a list of libraries to be linked with, like -lxml2 -lz -llzma -licui18n -licuuc -licudata -lm -ldl, and turns them it into four or five variables, that make their way into the generated Makefile. One of those is LD_RUN_PATH. It contains all the paths where the libraries where found.

$is_dyna is true if the library is dynamic (not ends with .a). $is_perl is true if Perl was not built with it, and %ld_run_path_seen is to not duplicate values in LD_RUN_PATH.

The part that calculates LD_RUN_PATH, that generates part of the Makefile with the variable, and the part that passes it to the linker.

Consequences

LD_RUN_PATH and in its turn RPATH entry in the binary make the loader search for libraries (at runtime) in the directories where they where found (at compile time).

-rpath=dir Add a directory to the runtime library search path. This is used when linking an ELF executable with shared objects. All -rpath arguments are concatenated and passed to the runtime linker, which uses them to locate shared objects at runtime. The -rpath option is also used when locating shared objects which are needed by shared objects explicitly included in the link; see the description of the -rpath-link option. If -rpath is not used when linking an ELF executable, the contents of the environment variable "LD_RUN_PATH" will be used if it is defined.

https://jlk.fjfi.cvut.cz/arch/manpages/man/core/binutils/ld.1.en

If a shared object dependency does not contain a slash, then it is searched for in the following order:

o Using the directories specified in the DT_RPATH dynamic section attribute of the binary if present and DT_RUNPATH attribute does not exist. Use of DT_RPATH is deprecated.

o Using the environment variable LD_LIBRARY_PATH, unless the executable is being run in secure-execution mode (see below), in which case this variable is ignored.

https://jlk.fjfi.cvut.cz/arch/manpages/man/core/man-pages/ld.so.8.en

$ perl Makefile.PL
$ make

Result,

$ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
 0x000000000000000f (RPATH)              Library rpath: [/usr/lib]

$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
  RPATH                /usr/lib

$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f6cfabb2000)

--dynamic - display contents of the .dynamic section, -x - display all headers.

Solution

One way to remedy this is to preload needed instance of libxml2,

$ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)

Another is to tell Makefile.PL search for libxml2 in the desired location (at compile time).

$ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build
$ make

Result,

$ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
 0x000000000000000f (RPATH)              Library rpath: [path/to/custom/libxml2/build/lib:/usr/lib]

$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
  RPATH                path/to/custom/libxml2/build/lib:/usr/lib

$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)

On a side note

Should you care to debug XML::LibXML, you can run make with OPTIMIZE variable (add -B to make make rebuild everything in case the library has already been built),

$ make OPTIMIZE='-g3 -O0'

There's also -O2, and no -g in LDDLFLAGS, but not sure if that matters much.

Alternatively, WriteMakeFile takes OPTIMIZE parameter, so you can add a line here,

'OPTIMIZE' => '-g3 -O0'

Or you can add the variable here, and pass it to Makefile.PL,

$ perl Makefile.PL OPTIMIZE='-g3 -O0'

Upvotes: 3

Nathan Mills
Nathan Mills

Reputation: 2279

If you install your custom built version of libxml in /usr/local/lib, I think the XML::LibXML module should load the custom one rather than the one in /usr/lib (Assuming you have sudoers or root permission to install it).

I'm not sure why Python seems to use a different lookup path for libraries than Perl does, though. Maybe the full path to libxml library is hardcoded in XML::LibXML's LibXML.so? If the XML::LibXML module was linked with -l/usr/lib/libxml2.so instead of -lxml2 that would be the problem.

Upvotes: -1

Related Questions