Unloading Java classes using CustomClassLoader

Question

I have been through many posts and questions but none of them had a absolute Java program to implement this.

Requirement : For some reasons my application loads the Common-codec 1.3.jar,

and later on ,within same jvm the process needs to use Common-code 1.10.jar which is of different version.

But since the previous classes are loaded and they have same package name ,reloading them with a Java program is not replacing the existing classes.

Here is the code (sample) i used to reload(replace) the existing keys,but found no luck as expected. Please let me know hoe this could be done by a Java example.

String pathToJar=/root/Desktop/commons-codec-1.10.jar";

            JarFile jarFile = null;
            try {
                jarFile = new JarFile(pathToJar);
            } catch (IOException e) {
                e.printStackTrace();
            }
            Enumeration<JarEntry> e = jarFile.entries();

            URL[] urls = new URL[0];
            try {
                urls = new URL[]{ new URL("jar:file:" + pathToJar+"!/") };
            } catch (MalformedURLException e1) {
                e1.printStackTrace();
            }
            URLClassLoader cl = URLClassLoader.newInstance(urls);

            while (e.hasMoreElements()) {
                JarEntry je = e.nextElement();
                if(je.isDirectory() || !je.getName().endsWith(".class")){
                    continue;
                }
                // -6 because of .class
                String className = je.getName().substring(0,je.getName().length()-6);
                className = className.replace(File.separatorChar, '.');
                String check="org.apache.commons.codec.binary.Base32";

                try {
                    Class c = cl.loadClass(className);   // Excepting it to replace old files,but thats not happening
                } catch (ClassNotFoundException e1) {
                    e1.printStackTrace();
                }

I was advised to write Custom class loader and unloading is possible via it. Could some show some relevant code or process of doing it.

Answer 1

Class "unloading" may only occur when both the Class and the ClassLoader that defined it are eligible for garbage collection. Furthermore, any given loader can only define a class of a specific name once. Therefore (save for exotic workarounds such as instrumentation or JDK implementation internals), there is no such thing as a "reloading-capable class loader" -- at least not in the true sense of the term; in order to obtain n distinct "versions" of a class, C, defineClass(C) must be invoked on n distinct ClassLoader objects.

Let's examine the following trivial use case: We have an app.jar, (unsurprisingly) containing a class declaring a main method; and two copies of a lib.jar, each having a single class, LibApiImpl, encapsulating a single method that just prints a version-dependent string to stdout. Our goal is to merely reference two copies of the "library" class from our main and see two different strings being output. The remainder of this post demonstrates just two of all the possible approaches to satisfy this requirement.

Approach #1 - Manually instantiating ClassLoaders

The straightforward solution is to simply create a new URLClassLoader every time a different LibApiImpl needs to be loaded. No custom class loader implementation is required, and the default parent-first delegation model serves both the application and the library well. A couple of things to note:

• The library JARs must not sit on the classpath, in order to not be discoverable by the default application class loader. Otherwise the application class loader will "favor" one over the other, and, due to the default delegation model, standard URLClassLoader children of the application loader will not be able to override their parent's opinion on the matter.
• To access the library class, the 3-argument Class::forName must be employed, specifying the manually-instantiated class loader. The single-arg version will delegate to the application class loader, which (as per the former point) is of course unaware of the existence of the corresponding .class file.
• If the reflectively-obtained class is to be cast to an interface, as will be the case herein, the interface must, as opposed to the implementation, reside on the classpath.

Demo code

package com.example.app;

import java.net.URL;
import java.net.URLClassLoader;

import com.example.lib.api.LibApi;

public class App {

    public static void main(String... args) throws Exception {
        try (URLClassLoader loader = new URLClassLoader(new URL[] { new URL("file:/path/to/lib1.jar") })) {
            ((LibApi) Class.forName("com.example.lib.impl.LibApiImpl", true, loader).newInstance()).printVersionInfo();
        }
        try (URLClassLoader loader = new URLClassLoader(new URL[] { new URL("file:/path/to/lib2.jar") })) {
            ((LibApi) Class.forName("com.example.lib.impl.LibApiImpl", true, loader).newInstance()).printVersionInfo();
        }
    }

}

---

package com.example.lib.api;

public interface LibApi {

    void printVersionInfo();

}

---

package com.example.lib.impl;

import com.example.lib.api.LibApi;

public class LibApiImpl implements LibApi {

    @Override
    public void printVersionInfo() {
        System.out.println("\n** lib " + getClass() + " / loaded by " + getClass().getClassLoader() + " **\n");
    }

}

Packaging

Correct the paths in com.example.app.App; then produce the following 4 JARs:

• An app.jar containing the com.example.app package.
• A lib-api.jar containing the com.example.lib.api package.
• Two "versions" (just export twice), lib1.jar and lib2.jar, each containing the com.example.lib.impl package.

Testing

Run as follows:

java -cp '/path/to/app.jar:/path/to/lib-api.jar' com.example.app.App

Sample output:

** lib class com.example.lib.impl.LibApiImpl / loaded by java.net.URLClassLoader@55f96302 **


** lib class com.example.lib.impl.LibApiImpl / loaded by java.net.URLClassLoader@135fbaa4 **

Approach #2 - Creating a "Master-Slave" loader

Sometimes the "parent-first" model is insufficient. As a contrived example, suppose that we wanted to obtain the "copy" of LibApiImpl that was last loaded by some child of the application class loader, without having to care about which child it actually was that defined the copy. In other words, we want the call Class.forName("com.example.lib.impl.LibApiImpl") to return the "most fresh" LibApiImpl version. But, unless we override the application class loader, or, more generally, the class loader of the App class, with a custom implementation, that call will always fail, because, under the default delegation model, delegation flows unidirectionally from lower-level loaders to their ancestors and not vice versa.

The application class loader implementation, given below, behaves as follows (refer to the Javadoc overview of the Loaders class for a concrete explanation): There is a "master" loader, serving as the application class loader, that may have a child, referred to as its "slave". The master is responsible for loading non-reloadable application classes from the classpath (app.jar and lib-api.jar, in this case), while the slave gets to load reloadable, non-classpath-residing ones (lib1.jar and lib2.jar). The communication between the two is bidirectional, and the loader that will ultimately define any given class, "fixed" or "reloadable", is always respectively the master and the slave, regardless the "initiating" loader, i.e., the loader on which the application called loadClass, or passed to Class::forName. Of course this is nothing but a toy implementation meant to (hopefully) illustrate how a different delegation scheme might look like, and is probably flawed in ways I have not even begun to imagine. A real-world implementation would, e.g., have to provide proper concurrency; provide compliant implementations of getResource et al.; address concerns of code accessibility, verification, and possibly code base privilege assignment; and allow extensibility by, and configuration of, multiple slaves, or even attachment of other children, potentially of arbitrary implementation, to the master, while preserving well-defined delegation semantics. The point of course being that it is generally too much effort to write a proper implementation, when there is OSGi and the like doing just that (among many others).

Demo code

package com.example.app;

import java.net.URL;


import com.example.app.Loaders.MasterLoader;
import com.example.lib.api.LibApi;

public class App2 {

    public static void main(String... args) throws Exception {

        MasterLoader loader = (MasterLoader) ClassLoader.getSystemClassLoader();
        loader.setDebug(true);
        loader.refresh(new URL[] { new URL("file:/path/to/lib1.jar") });

        newLibApi().printVersionInfo();

        loader.refresh(new URL[] { new URL("file:/path/to/lib2.jar") });

        newLibApi().printVersionInfo();

    }

    static LibApi newLibApi() throws Exception {
        return (LibApi) Class.forName("com.example.lib.impl.LibApiImpl").newInstance();
    }

}

---

package com.example.app;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Paths;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.regex.Pattern;

/**
 * Contains a "Master (parent)/Slave (child)" <code>ClassLoader</code> "duo".<br/>
 * <br/>
 * The class loading "protocol" adhered by the respective {@code loadClass(String, boolean)}
 * implementations of the two is as follows:
 * <ol>
 * <li>If the name argument matches the name of the loader's class itself, return that class.
 * Otherwise proceed.</li>
 * <li>If a call to {@code findLoadedClass(name)} yields a non-null class, return that class.
 * Otherwise proceed.</li>
 * <li>Let <em>C</em> be either this loader's parent, if this is a "slave", or its child, if this is
 * a "master". If <em>C</em> is non-null, have it execute steps (1), (2) itself. If a class gets
 * produced, return that class. Otherwise (i.e., if there either is no result, or <em>C</em> is
 * null) proceed.</li>
 * <li>If the name argument refers to a potential bootstrap classpath class name, call
 * {@code loadClass(name)} on the default system classloader (the "master's" parent). If the call
 * succeeds, return that class. Otherwise proceed.</li>
 * <li>If the name argument refers to a .class file under this loader's search path, read it, define
 * and link a new class off of its contents, and return that "freshly-fefined" class. Otherwise
 * proceed.</li>
 * <li>Once again, let <em>C</em> be the loader specified in step (3). If non-null, have it execute
 * step (5) on itself. If a class gets produced, return that class. Otherwise fail.</li>
 * </ol>
 */
public class Loaders {

    private static class SlaveLoader extends URLClassLoader {

        static final Pattern BOOT_CLASS_PATH_RES_NAMES = Pattern.compile("((com\\.)?sun|java(x)?)\\..*");
        static final URL[] EMPTY_SEARCH_PATH = new URL[0];

        static final URL[] createSearchPath(String pathNames) {
            if (pathNames != null) {
                List<URL> searchPath = new ArrayList<>();
                for (String pathName : pathNames.split(File.pathSeparator)) {
                    try {
                        searchPath.add(Paths.get(pathName).toUri().toURL());
                    }
                    catch (MalformedURLException e) {
                        e.printStackTrace();
                    }
                }
                return searchPath.toArray(new URL[0]);
            }
            return EMPTY_SEARCH_PATH;
        }

        static final byte[] readClassData(URL classResource) throws IOException {
            try (InputStream in = classResource.openStream(); ByteArrayOutputStream out = new ByteArrayOutputStream()) {
                while (in.available() > 0) {
                    out.write(in.read());
                }
                return out.toByteArray();
            }
        }

        final String loadClassOutcomeMsgFmt = "loadClass return '{'\n\tloader = {0}\n\ttarget = {1}\n\tresult : {2}\n'}'";
        volatile boolean debug;

        SlaveLoader(URL[] searchPath, ClassLoader parent) {
            super(searchPath, parent);
        }

        @Override
        protected synchronized Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
            validateName(name);
            Class<?> ret = loadFromCache(name);
            if (ret != null) {
                return ret;
            }
            MasterLoader parent = (MasterLoader) getParent();
            if ((ret = parent.loadFromCache(name)) != null) {
                log(loadClassOutcomeMsgFmt, this, name, "3 - early return - pre-loaded/cached - via " + parent);
                return ret;
            }
            if ((ret = loadFromBootClasspath(name)) != null) {
                return ret;
            }
            if ((ret = loadFromSearchPath(name, resolve)) != null) {
                return ret;
            }
            if ((ret = parent.loadFromSearchPath(name, resolve)) != null) {
                log(loadClassOutcomeMsgFmt, this, name,
                        "6 - common/non-reloadable classpath delegation - via " + parent);
                return ret;
            }
            if ((ret = parent.loadFromSearchPath(name, resolve)) != null) {
                return ret;
            }
            throw createCnfe(name, null);
        }

        void validateName(String name) throws ClassNotFoundException {
            log("loadClass entry '{'\n\tloader = {0}\n\ttarget = {1}\n'}'", this, name);
            if ((name == null) || name.trim().isEmpty()) {
                throw createCnfe(name, null);
            }
        }

        Class<?> loadFromCache(String name) {
            Class<?> ret = getClass();
            if (ret.getName().equals(name)) {
                log(loadClassOutcomeMsgFmt, this, name, "1 - early return - own class");
                return ret;
            }
            if ((ret = findLoadedClass(name)) != null) {
                log(loadClassOutcomeMsgFmt, this, name, "2 - early return - pre-loaded/cached");
                return ret;
            }
            return null;
        }

        Class<?> loadFromBootClasspath(String name) {
            if (BOOT_CLASS_PATH_RES_NAMES.matcher(name).matches()) {
                ClassLoader defSysCl = ClassLoader.getSystemClassLoader().getParent();
                try {
                    Class<?> ret = ClassLoader.getSystemClassLoader().getParent().loadClass(name);
                    log(loadClassOutcomeMsgFmt, this, name, "4 - bootstrap classpath delegation - via " + defSysCl);
                    return ret;
                }
                catch (ClassNotFoundException cnfe) {
                }
            }
            return null;
        }

        Class<?> loadFromSearchPath(String name, boolean resolve) throws ClassNotFoundException {
            Class<?> ret = null;
            URL res = findResource(name.replace(".", "/") + ".class");
            if (res != null) {
                byte[] b;
                try {
                    b = readClassData(res);
                }
                catch (IOException ioe) {
                    throw createCnfe(name, ioe);
                }
                ret = defineClass(name, b, 0, b.length);
                if (resolve) {
                    resolveClass(ret);
                }
                log(loadClassOutcomeMsgFmt, this, name, "5 - freshly-defined from local search path");
                return ret;
            }
            return null;
        }

        ClassNotFoundException createCnfe(String name, Throwable cause) throws ClassNotFoundException {
            return new ClassNotFoundException(MessageFormat.format("Class loading : {0} : {1} : FAILED", this,
                    (name == null) ? "null" : name, cause));
        }

        void log(String msg, Object... args) {
            if (debug) {
                System.out.println(MessageFormat.format("\n" + msg + "\n", args));
            }
        }

        public void setDebug(boolean debug) {
            this.debug = debug;
        }

        @Override
        protected void finalize() throws Throwable {
            try {
                close();
            }
            finally {
                super.finalize();
            }
            log("ClassLoader finalization : {0}", this);
        }

    }

    public static class MasterLoader extends SlaveLoader {

        static final URL[] DEFAULT_CLASS_PATH = createSearchPath(System.getProperty("java.class.path"));

        private URL[] reloadableSearchPath = EMPTY_SEARCH_PATH;
        private volatile SlaveLoader slave;

        public MasterLoader(ClassLoader parent) {
            super(DEFAULT_CLASS_PATH, parent);
        }

        public synchronized void refresh(URL[] reloadableSearchPath) {
            int len;
            if ((reloadableSearchPath != null) && ((len = reloadableSearchPath.length) > 0)) {
                List<URL> path = new ArrayList<>(len + 1);
                for (int i = 0; i < len; i++) {
                    URL entry = reloadableSearchPath[i];
                    if (entry != null) {
                        path.add(entry);
                    }
                }
                this.reloadableSearchPath = (!path.isEmpty()) ? path.toArray(EMPTY_SEARCH_PATH) : EMPTY_SEARCH_PATH;
            }
            else {
                this.reloadableSearchPath = EMPTY_SEARCH_PATH;
            }
            if (slave != null) {
                try {
                    slave.close();
                }
                catch (IOException ioe) {
                }
                slave = null;
                /*
                 * At least two calls to System::gc appear to be required in order for Class::forName to cease
                 * returning cached classes previously defined by slave and for which the master served as an
                 * intermediary, i.e., an "initiating loader".
                 * 
                 * See also http://blog.hargrave.io/2007/09/classforname-caches-defined-class-in.html
                 */
                for (int i = 0; i < 2; i++) {
                    System.gc();
                    try {
                        Thread.sleep(100);
                    }
                    catch (InterruptedException ie) {
                    }
                }
            }
            if (this.reloadableSearchPath != EMPTY_SEARCH_PATH) {
                log("Class loader search path refresh : {0}\n\tSearch path = {1}", this,
                        Arrays.toString(this.reloadableSearchPath));
                slave = new SlaveLoader(this.reloadableSearchPath, this);
                slave.setDebug(debug);
            }
        }

        @Override
        protected synchronized Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
            validateName(name);
            Class<?> ret = loadFromCache(name);
            if (ret != null) {
                return ret;
            }
            if ((slave != null) && ((ret = slave.loadFromCache(name)) != null)) {
                log(loadClassOutcomeMsgFmt, this, name, "3 - early return - pre-loaded/cached - via " + slave);
                return ret;
            }
            if ((ret = loadFromBootClasspath(name)) != null) {
                return ret;
            }
            if ((ret = loadFromSearchPath(name, resolve)) != null) {
                return ret;
            }
            if ((slave != null) && ((ret = slave.loadFromSearchPath(name, resolve)) != null)) {
                log(loadClassOutcomeMsgFmt, this, name,
                        "6 - reloadable classpath delegation - via " + ret.getClassLoader());
                return ret;
            }
            throw createCnfe(name, null);
        }

    }

}

---

Packaging

Once again correct the paths in com.example.app.App2; add App2 and com.example.app.Loaders to app.jar; and re-export. The other JARs should remain as in the former example.

Testing

Run as follows:

java -cp '/path/to/app.jar:/path/to/lib-api.jar' \
'-Djava.system.class.loader=com.example.app.Loaders$MasterLoader' \
com.example.app.App2

Sample output (loadClass debug omitted):

** lib class com.example.lib.impl.LibApiImpl / loaded by com.example.app.Loaders$SlaveLoader@7f31245a **

...

ClassLoader finalization : com.example.app.Loaders$SlaveLoader@7f31245a

...

** lib class com.example.lib.impl.LibApiImpl / loaded by com.example.app.Loaders$SlaveLoader@12a3a380 **