Java: load a library that depends on other libs

Question

I want to load my own native libraries in my java application. Those native libraries depend upon third-party libraries (which may or may not be present when my application is installed on the client computer).

Inside my java application, I ask the user to specify the location of dependent libs. Once I have this information, I am using it to update the "LD_LIBRARY_PATH" environment variable using JNI code. The following is the code snippet that I am using to change the "LD_LIBRARY_PATH" environment variable.

Java code

public static final int setEnv(String key, String value) {
        if (key == null) {
            throw new NullPointerException("key cannot be null");
        }
        if (value == null) {
            throw new NullPointerException("value cannot be null");
        }
        return nativeSetEnv(key, value);
    }

public static final native int nativeSetEnv(String key, String value);

Jni code (C)

    JNIEXPORT jint JNICALL Java_Test_nativeSetEnv(JNIEnv *env, jclass cls, jstring key, jstring value) {
    const char *nativeKey = NULL;
    const char *nativeValue = NULL;
    nativeKey = (*env)->GetStringUTFChars(env, key, NULL);
    nativeValue = (*env)->GetStringUTFChars(env, value, NULL);
    int result = setenv(nativeKey, nativeValue, 1);
    return (jint) result;
}

I also have corresponding native methods to fetch the environment variable.

I can successfully update the LD_LIBRARY_PATH (this assertion is based on the output of C routine getenv().

I am still not able to load my native library. The dependent third-party libraries are still not detected.

Any help/pointers are appreciated. I am using Linux 64 bit.

Edit:

I wrote a SSCE (in C) to test if dynamic loader is working. Here is the SSCE

#include 
#include 
#include 
#include 

int main(int argc, const char* const argv[]) {

    const char* const dependentLibPath = "...:";
    const char* const sharedLibrary = "...";
    char *newLibPath = NULL;
    char *originalLibPath = NULL;
    int l1, l2, result;
    void* handle = NULL;

    originalLibPath = getenv("LD_LIBRARY_PATH");
    fprintf(stdout,"\nOriginal library path =%s\n",originalLibPath);
    l1 = strlen(originalLibPath);
    l2 = strlen(dependentLibPath);
    newLibPath = (char *)malloc((l1+l2)*sizeof(char));
    strcpy(newLibPath,dependentLibPath);
    strcat(newLibPath,originalLibPath);
    fprintf(stdout,"\nNew library path =%s\n",newLibPath);

    result = setenv("LD_LIBRARY_PATH", newLibPath, 1);
    if(result!=0) {
        fprintf(stderr,"\nEnvironment could not be updated\n");
        exit(1);
    }    
    newLibPath = getenv("LD_LIBRARY_PATH");
    fprintf(stdout,"\nNew library path from the env =%s\n",newLibPath);

    handle = dlopen(sharedLibrary, RTLD_NOW);
    if(handle==NULL) {
        fprintf(stderr,"\nCould not load the shared library: %s\n",dlerror());
        exit(1);        
    }
    fprintf(stdout,"\n The shared library was successfully loaded.\n");

    result = dlclose(handle);
    if(result!=0) {
        fprintf(stderr,"\nCould not unload the shared library: %s\n",dlerror());
        exit(1);
    }

    return 0;
}

The C code also does not work. Apparently, the dynamic loader is not rereading the LD_LIBRARY_PATH environment variable. I need to figure out how to force the dynamic loader to re-read the LD_LIBRARY_PATH environment variable.

Answer 1

This is a hack used to manipulate JVM's library path programmatically. NOTE: it relies on internals of ClassLoader implementation so it might not work on all JVMs/versions.

String currentPath = System.getProperty("java.library.path");
System.setProperty( "java.library.path", currentPath + ":/path/to/my/libs" );

// this forces JVM to reload "java.library.path" property
Field fieldSysPath = ClassLoader.class.getDeclaredField( "sys_paths" );
fieldSysPath.setAccessible( true );
fieldSysPath.set( null, null );

This code uses UNIX-style file path separators ('/') and library path separator (':'). For cross-platform way of doing this use System Properties to get system-specific separators: http://download.oracle.com/javase/tutorial/essential/environment/sysprop.html

Answer 2

See the accepted answer here:

Changing LD_LIBRARY_PATH at runtime for ctypes

In other words, what you're trying to do isn't possible. You'll need to launch a new process with an updated LD_LIBRARY_PATH (e.g., use ProcessBuilder and update environment() to concatenate the necessary directory)

Answer 3

I have successfully implemented something similar for CollabNet Subversion Edge, which depends on the SIGAR libraries across ALL Operating Systems (we support Windows/Linux/Sparc both 32 bits and 64 bits)...

Subversion Edge is a web application that helps one managing Subversion repositories through a web console and uses SIGAR to the SIGAR libraries helps us provide users data values directly from the OS... You need to update the value of the property "java.library.path" at runtime. (https://ctf.open.collab.net/integration/viewvc/viewvc.cgi/trunk/console/grails-app/services/com/collabnet/svnedge/console/OperatingSystemService.groovy?revision=1890&root=svnedge&system=exsy1005&view=markup Note that the URL is a Groovy code, but I have modified it to a Java here)...

The following example is the implementation in URL above... (On Windows, your user will be required to restart the machine if he/she has downloaded the libraries after or downloaded them using your application)... The "java.library.path" will update the user's path "usr_paths" instead of System path "sys_paths" (permissions exception might be raised depending on the OS when using the latter).

133/**
134 * Updates the java.library.path at run-time.
135 * @param libraryDirPath
136 */
137 public void addDirToJavaLibraryPathAtRuntime(String libraryDirPath) 
138    throws Exception {
139    try {
140         Field field = ClassLoader.class.getDeclaredField("usr_paths");
141         field.setAccessible(true);
142         String[] paths = (String[])field.get(null);
143         for (int i = 0; i < paths.length; i++) {
144             if (libraryDirPath.equals(paths[i])) {
145                 return;
146             }
147         }
148         String[] tmp = new String[paths.length+1];
149         System.arraycopy(paths,0,tmp,0,paths.length);
150         tmp[paths.length] = libraryDirPath;
151         field.set(null,tmp);
152         String javaLib = "java.library.path";
153         System.setProperty(javaLib, System.getProperty(javaLib) +
154             File.pathSeparator + libraryDirPath);
155 
156     } catch (IllegalAccessException e) {
157         throw new IOException("Failed to get permissions to set " +
158             "library path to " + libraryDirPath);
159     } catch (NoSuchFieldException e) {
160         throw new IOException("Failed to get field handle to set " +
161            "library path to " + libraryDirPath);
162     }
163 }

The Bootstrap services (Groovy on Grails application) class of the console runs a service and executes it with the full path to the library directory... UNiX-based servers do not need to restart the server to get the libraries, but Windows boxes do need a server restart after the installation. In your case, you would be calling this as follows:

     String appHomePath = "/YOUR/PATH/HERE/TO/YOUR/LIBRARY/DIRECTORY";
     String yourLib = new File(appHomePath, "SUBDIRECTORY/").getCanonicalPath();
124  try {
125      addDirToJavaLibraryPathAtRuntime(yourLib);
126  } catch (Exception e) {
127      log.error("Error adding the MY Libraries at " + yourLib + " " +
128            "java.library.path: " + e.message);
129  }

For each OS you ship your application, just make sure to provide a matching version of the libraries for the specific platform (32bit-Linux, 64bit-Windows, etc...).