OSError: cannot open shared object file: No such file or directory even though file is in the folder

Question

I've been fighting with this for quite some time now. I'm trying to install Yaafe for audio feature extraction. I follow instructions here: https://github.com/Yaafe/Yaafe

Everything installs up nicely, but when I try to run the test file "frames.py" I get following error:

  File "frames.py", line 6, in <module>
    from yaafelib import FeaturePlan, Engine, AudioFileProcessor 
  File "/usr/local/lib/python2.7/dist-packages/yaafelib/__init__.py", line 36, in <module>
    from yaafelib.core import (loadComponentLibrary,
  File "/usr/local/lib/python2.7/dist-packages/yaafelib/core.py", line 35, in <module>
    yaafecore = cdll.LoadLibrary('libyaafe-python.so')
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libyaafe-python.so: cannot open shared object file: No such file or directory

I have included the lib directory to LD_LIBRARY_PATH with following command:

export LD_LIBRARY_PATH=/usr/local/lib

And indeed when I echo the LIBRARY_PATH it is there. Also when I check the /usr/local/lib it has following contents:

libyaafe-components.so         libyaafe-io.so             python2.7
libyaafe-components.so.0       libyaafe-io.so.0           python3.4
libyaafe-components.so.0.70.0  libyaafe-io.so.0.70.0      site_ruby
libyaafe-core.so               libyaafe-python.so         yaafe
libyaafe-core.so.0             libyaafe-python.so.0
libyaafe-core.so.0.70.0        libyaafe-python.so.0.70.0

So shouldn't everything be okay? I don't understand what is the problem. I've followed instructions to the point.

Answer 1

I know this post is old, but if you cannot modify (or do not want to modify) LD_LIBRARY_PATH and you still need a way to load the shared object into memory in Python, then you can use this general approach.

import ctypes
ctypes.CDLL('/usr/local/lib/library.version.so', mode=ctypes.RTLD_GLOBAL)

For you, the call might look like:

import ctypes
ctypes.CDLL('/usr/local/lib/libyaafe-python.so', mode=ctypes.RTLD_GLOBAL)

And if you don't know the path beforehand, I've put together the following function that can recursively search a set of library paths for a specified library name (and version). Hope this helps!

# Given a library name, try to locate the library and load it in
#  global mode as a CDLL so that it is accessible to all code.
def load_library(libname, version=None, paths=("/usr/lib","/usr/local/lib",),
                 extensions=(".so",), prefixes=("","lib",),
                 recursive=True):
    import os, ctypes
    # This code will find all possible matches for the library, 
    #  and pick the first lexicographically.
    versions = set()
    for directory in paths:
        # Enumerate all files at that path (depending on "resursive" option).
        if recursive:
            file_paths = sum((
                [os.path.join(dirpath, f) for f in filenames]
                for (dirpath, dirnames, filenames) in os.walk(directory)
            ), [])
        else:
            file_paths = [
                os.path.join(directory,f) for f in os.listdir(directory)
                if not os.path.isdir(f)
            ]
        # Iterate over the files looking for the specified library.
        for path in file_paths:
            filename = os.path.basename(path)
            # Verify the file extension is allowed.
            ext = filename[len(filename)-filename[::-1].find('.')-1:] 
            # Examples of the value for 'ext' given a 'filename':
            #   "foo.bar" -> ".bar", "foobar" -> ""
            if (ext not in extensions): continue
            # Check that the library name is in file name.
            if (libname not in filename): continue
            # Check that the version is in the name (if it is specified).
            file_version = ".".join(filename.split('.')[1:-1])
            if ((version is not None) and (version != file_version)): continue
            # Extract the file name and check for matches against prefixes.
            name = filename[:(filename+'.').find('.')]
            for p in prefixes:
                if (p+libname == name): break
            else: continue
            # Now this filename matches the required:
            #   name preceding the first ".",
            #   file extension including last ".",
            #   version between first and last "." if that was specified,
            #   and it exists in one of the provided paths.
            versions.add(path)
            # Uncomment the following line to see the considered libraries.
            # print([path, filename, name, file_version, ext, os.path.islink(path)])
    # Raise an error if no versions could be found.
    if (len(versions) == 0):
        raise(FileNotFoundError(f"No library file found for '{libname}'{'' if version is None else ' version '+version}."))
    # Get the library path as the first element of the set of discovered versions.
    library_path = sorted(versions)[0]
    # Load the library globally (so it is accessible to later codes) and return its path.
    ctypes.CDLL(library_path, mode=ctypes.RTLD_GLOBAL)
    return library_path

And for your specific example the call to the function would look like:

load_library("yaafe-python")

As long as you called this before importing whatever module was failing to load, everything should work.

WARNINGS

• This is written for POSIX systems (Ubuntu, macOS, ...)
• This loads the library into the global C namespace, so I am not sure how it will handle duplicate declarations if something already exists and it might cause unexpected downstream consequences if a loaded library is overwritten.

Answer 2

Change your code so that you print os.environ right before that exception occurs. That way you will see whether the Python process has the correct environment set or not. The other obvious thing to check are whether your Python process has sufficient permission to open and read libyaafe-python.so. Note that sudo by default limits the environment of the invoked command, for security reasons (see here, for instance).