Why is this lambda function initiating class loading in the wrong classloader?

Question

I am working on an application that has a plugin framework. It uses a custom classloader to load plugins from encrypted jar files. Initially I painted myself into using the custom classloader as a bootstrapped system classloader. In this way it worked, but there are some drawbacks in using a custom system classloader.

I am trying to rework so that the custom classloader is only utilized to load the plugins. The plugins are hierarchical in nature and therefore need the same class context. To that end, the plugin classloader CustomClassloader is a singleton which extends ClassLoader and has the parent classloader set to the SystemClassloader (and delegates classloading to parent as is the normal pattern).

This seems to be working well EXCEPT in a particular case where I need to create a lambda function that allows the generic ('reflective') setting of a POJO boolean field that is defined within the plugin.

lambda_set creation (defined within an application jar that is loaded by the system classloader):

private BiConsumer<POJO_Interface, Object> lambda_set = null;
Class[] parameter = new Class[1];

parameter[0] = field_clazz; // in this case it is boolean.class
set_method = pojo_class.getMethod(setter.trim(), parameter); // setter method name
set_method.setAccessible(true);

MethodHandles.Lookup lookup = MethodHandles.lookup();
MethodHandle target = lookup.unreflect(set_method);
MethodType func = target.type();
MethodType func1 = func.changeParameterType(0, Object.class);
if(func.parameterCount() >= 2)
    func1 = func1.changeParameterType(1, Object.class);

CallSite site = LambdaMetafactory.metafactory(lookup, "accept", 
    MethodType.methodType(BiConsumer.class), func1, target, func);
MethodHandle factory = site.getTarget();
lambda_set = (BiConsumer) factory.invoke();

When I call lambda_set.accept(pojo, value); I get a ClassNotFoundException for the POJOs superclass. Each POJO extends it's own parent abstract class that implements the POJO_Interface and contains its fields and getters/setters. This same function works fine when everything is loaded from the custom bootstrap classloader. I've verified that it is trying to load the POJO's parent class in the System classloader exclusively instead of the CustomClassloader which is wrong.

I've verified that the pojo.getClass().getClassLoader() == pojo_class.getClassLoader() == CustomClassloader.class However, the lambda_set.getClass().getClassLoader() == jdk.internal.loader.ClassLoaders$AppClassLoader. I'm not sure if this is the problem.

This behavior is the same in JDK8-JDK14.

Is there a way that I can make the lambda_set utilize my CustomClassloader when it needs to load a class? Any other insights would be appreciated!

I've also tried setting the application main thread ContextClassloader and verified that the lambda_set is being called from a thread who's ContextClassLoader is the CustomClassloader. This results in the same behavior desicribed above.

    static {
        Thread.currentThread().setContextClassLoader(new CustomClassloader(ClassLoader.getSystemClassLoader()));
    }

public static void main(String[] args) {...}

Answer 1

Based on Holger's comment:

{...} You are passing the result of MethodHandles.lookup() which encapsulates the context in which the MethodHandles.lookup() expression is contained. The fix is to provide a lookup representing a context which can resolve the type, e.g. encapsulating the target method’s declaring class. – Holger

And additionally digging through the MethodHandles source code some we see clearly what Holger is referring to:

    /**
     * Returns a {@link Lookup lookup object} with
     * full capabilities to emulate all supported bytecode behaviors of the caller.
     {...}
     */
    @CallerSensitive
    @ForceInline // to ensure Reflection.getCallerClass optimization
    public static Lookup lookup() {
        return new Lookup(Reflection.getCallerClass());
    }

The call to Reflection.getCallerClass()) is what is tying the context to the class from the wrong classloader.

This led me to look for an alternative, and one presented itself within the MethodHanldes source comments/methods:

    /**
     * Returns a {@link Lookup lookup} object on a target class to emulate all supported
     * bytecode behaviors, including <a href="MethodHandles.Lookup.html#privacc">private access</a>.
     {...}
     */
    public static Lookup privateLookupIn(Class<?> targetClass, Lookup caller) 
        throws IllegalAccessException {...}

With this understanding, I have updated the code from the question to the following:

private BiConsumer<POJO_Interface, Object> lambda_set = null;
Class[] parameter = new Class[1];

parameter[0] = field_clazz; // in this case it is boolean.class
set_method = pojo_class.getMethod(setter.trim(), parameter); // setter method name
set_method.setAccessible(true);

MethodHandles.Lookup lookup = MethodHandles.lookup();

/////// FIXED //////////
lookup = MethodHandles.privateLookupIn(pojo_class, lookup);
/////// FIXED //////////

MethodHandle target = lookup.unreflect(set_method);
MethodType func = target.type();
MethodType func1 = func.changeParameterType(0, Object.class);
if(func.parameterCount() >= 2)
    func1 = func1.changeParameterType(1, Object.class);

CallSite site = LambdaMetafactory.metafactory(lookup, "accept", 
    MethodType.methodType(BiConsumer.class), func1, target, func);
MethodHandle factory = site.getTarget();
lambda_set = (BiConsumer) factory.invoke();

Fortunately, in my case everything is under the same module name, so I was able to utilize the existing lookup in the call to MethodHandles.privateLookupIn(Class<?> targetClass, Lookup caller).

With the change, this function now works correctly. Once again, thank you Holger for steering me in the right direction.

For additional information, Holger has also answered related questions:

LambdaMetafactory to access class on a different ClassLoader and Use LambdaMetafactory to invoke one-arg method on class instance obtained from other classloader (for Java 8)