Metaspace memory leak caused by hot loading

Question

The application is written in java 8 and the app is run on 1.8.0_362,1.8.0_371.

I use custom classloader to hot load classes(let's call script classes), which are under the package com.xxx.project1.script

My custom classloader is:

public class ScriptClassLoader extends URLClassLoader {
    private String classPackage;
    private ClassLoader defaultClassLoader;
    private boolean dev;

    public static ScriptClassLoader newInstance(URL[] urls, String classPackage, boolean dev) {
        if (dev) {
            urls = ((URLClassLoader)ScriptClassLoader.class.getClassLoader()).getURLs();
        }

        return new ScriptClassLoader(urls, classPackage, dev);
    }

    protected ScriptClassLoader(URL[] urls, String classPackage, boolean dev) {
        super(urls);
        this.classPackage = classPackage;
        this.dev = dev;
        this.defaultClassLoader = ScriptClassLoader.class.getClassLoader();
    }

    protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
        if (!this.dev && name.startsWith(this.classPackage)) {
            Class<?> c = this.findLoadedClass(name);
            if (c == null) {
                c = this.findClass(name);
            } 

            if (resolve) {
                this.resolveClass(c);
            }

            return c;
        } else {
            return super.loadClass(name, resolve);
        }
    }
}

Recently I found after reload script classes, the original script classes didn't get unloaded. The reason is when creating a new thread, it will hold a new context object(see java.lang.Thread):

this.inheritedAccessControlContext =
                acc != null ? acc : AccessController.getContext();

So If a script class is created by such a thread, the object will hold the ScriptClassLoader object used to load the script class, causing this ScriptClassLoader to be unable to be GC and thus unable to unload useless script classes.

The gc root strace:

com.xxx.engine.script.ScriptClassLoader
-classloader java.security.ProtectionDomain
--[4] java.security.ProtectionDomain[7]
---context java.security.AccessControlContext
----inheritedAccessControlContext java.lang.Thread

class com.xxx.project1.script.impl.AdvanceXXXScript
-java.lang.Object[160]
--elementData java.util.Vector
---classes com.xxx.engine.script.ScriptClassLoader
----classloader java.security.ProtectionDomain
-----[4]java.security.ProtectionDomain[7]
------context java.security.AccessControlContext
-------inheritedAccessControlContext java.lang.Thread

Since I'm not clear about the use of inheritedAccessControlContext and it cannot be accessed from external class, the current solution is to initialize all core threads when creating the thread pool, avoiding thread creation triggered by script classes:

public static ScheduledExecutorService newScheduledThreadPool(int coreSize, String namePrefix) {
        ScheduledExecutorService service = Executors.newScheduledThreadPool(coreSize, new ThreadFactory() {
            final AtomicInteger n = new AtomicInteger();
            @Override
            public Thread newThread(Runnable r) {
                String name = namePrefix + "-" + n.incrementAndGet());
                return new Thread(r, name);
            }
        });
        ((ScheduledThreadPoolExecutor) service).prestartAllCoreThreads(); // initialize all core threads

        return service;
    }

But there are still some hidden dangers. Because Java only loads classes when they are in use. Therefore, in some places, even if static constants are written, it is still possible for script classes to trigger creation, such as:

abstract public class Http {
    private static final ScheduledExecutorService EXECUTOR = Executors.newScheduledThreadPool(16,
            new ThreadFactory() {
                private final AtomicInteger counter = new AtomicInteger(0);

                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, "http-request-" + counter.incrementAndGet());
                }
            }
    );

If it is the first time using this class in a script class, then it will undoubtedly return to the above problem. The current solution is to proactively call these class/methods outside of the script during program startup to trigger their initialization, for example:

private static void envirInit() {
    Http.init();
    DBUpdateUtil.init();
}

It seems stupid. Is there a better solution? Thanks in advance for any help.

---

UPDATE:

I found setting classloader in ProtectionDomain in inheritedAccessControlContext to null works, but get no idea if it will cause any problem.

    ClassLoader classLoader = RoleScript.getClass().getClassLoader(); // RoleScript is under the package:com.xxx.project1.script
    try {
        Vector<Class<?>> classes = (Vector<Class<?>>) org.apache.commons.lang3.reflect.FieldUtils.readDeclaredField(classLoader, "classes", true);
        for (Class<?> aClass : classes) {
            FieldUtils.writeDeclaredField(aClass.getProtectionDomain(), "classloader", null, true);
        }
    } catch (IllegalAccessException e) {
        throw new RuntimeException(e);
    }

---

The subsequent post: Does metaspace gc has some relevance with different MaxMetaspaceSize value?