How do I cache Method objects related to a Class if the cache must not keep references

Question

I am in a situation where I have a relatively expensive operation to determine a certain subset of the methods on a java class. In order to optimize, I'd like to keep a cache, sort of like this:

private final static HashMap<Class<?>, Set<Method>> cache = new HashMap<>();

However, I am also in long running server environment, where we would like classloaders to come and go. The above cache is no good because it will hold onto the classes, preventing the classloaders from being garbage collected.

My first try at a fix was:

private final static WeakHashMap<Class<?>, Set<Method>> cache = new HashMap<>();

Unfortunately, this does not work either, since the Method objects in the set keep a hard-reference back to the Class, implying that the point of the WeakHashMap is lost.

I have tried several other things. For example, I've defined a data structure where the held Method object was a WeakReference. I get no love there either, because while the Method holds a hard reference back to the Class, the Class does not in fact hold a reference to the Method, meaning that my WeakReference to the method often returns a null from the get() method (if no-one else ends up holding onto one of the Methods in the set).

My specific question is what is a good way of accomplishing this cache from Class to set of Methods while not keeping any hard references to the Class?

Answer 1

Edit: I've now implemented the most fancy of my own suggestions below – using WeakReference and injecting a class with a strong reference into the ClassLoader owning the cached results: https://github.com/Legioth/reflectioncache

---

I've been researching the same kind of issue and I've found a couple of approaches, each with their own pros and cons. I'll describe these from the point of view of a library that needs to cache its results.

Strong references

Even if you keep strong references to the Method instances, there are some ways of dealing with the classloader leak issue.

Selective caching

Only cache values from the same classloader that loaded the class containing the cache. In most server environments, this means that the library doing the caching should be in the same .war as the code that uses the library, except if OSGi is also involved. For classes that can't be cached, you could calculate the value every time or throw an exception.

Verify classloader after cache hit

A fragile approach that might work in some specific situations is to use e.g. the Class.getName() string as the cache key and verify that the result originates from the right classloader before using the old value. The trick here is to replace the old cache entry with a new one if the classloader has changed. Eventually all old entries will have been refreshed in this way, assuming the same keys will still be used after a redeploy, and assuming that there's always a redeploy, never just stopping to use the cache.

Explicit eviction

Require users of the library to keep track of when their classloaders are about to be discarded and inform the library to clear everything cached for that classloader. An example of this pattern can be seen in java.util.ResourceBundle.clearCache(ClassLoader), although the class also provides other cache management mechanisms.

Timed eviction

The last approach involving strong references that I've found is to remove cache entries that haven't been used for a while. Depending on the nature of the cached data, you could either clear out individual expired entires, or alternatively just keep one timestamp for each encountered classloader and evict all entries associated with that classloader. Simple eviction can be done each time the cache is used, but this approach will always leak the classloader of the last user since there won't be anyone using the cache after that. Working around that issue requires a timer thread, which could by itself also be a source of memory leaks unless carefully managed.

SoftReference

The JVM (at least the Oracle / OpenJDK version) already provides something similar to the approach with evicting when an entry hasn't been used for a time: SoftReference. Documentation about SoftRefLRUPolicyMSPerMB implies that a SoftReference is reclaimed based on how long time it is since it has last been accessed, adjusted by the amount of available memory. The main drawback with this approach is that there will be many cache misses if the system runs close to its memory limit.

WeakReference

As you have already found out, directly using a WeakReference to the Method does not work since there's then nothing that prevents the Method instance from being collected. To prevent the Method instance from being collected, you'd need to make sure there is at least one strong reference to it. To prevent that strong reference from causing a classloader leak, it must originate from the classloader of the class defining the Method. The most straightforward source of such a reference would be through a static field in a class loaded by that classloader. Once you have that one field, you can store the actual cache map for values from that classloader in it. The class managing the cache could then use a WeakReference to the actual map.

User-provided reference holder

Depending on the nature of the library, it might be feasible to require the library user to provide a class with such a static field for the library to (ab)use.

Generated reference holder

One potential way of creating this reference would be to use reflection to run ClassLoader.defineClass with bytecode for a simple class that only contains one static field, and then use more reflection to update the value of that field as described above. I haven't tried this out in practice, but if actually does work, it seems like it would be the holy grail of classloader caching.

Answer 2

Hmm, I have a bunch of ideas:

First: Regarding the key you could use the class name instead of the class object. Then you validate that the resulting methods belong to the correct class / class loader.

class CacheEntry { Set<Method> methods; Class<?> klass; }
Cache<String, CacheEntry> cache = ...

Set<Method> getCachedMethods(Class<?> c) {
  CacheEntry e = cache.get(klass.getName());
  if ((e != null && e.klass != c) ||
      e == null) {
    e = recomputeEntry(c);
    cache.put(c, e);
  }
  return e.methods;
}

Okay, this won't work if you regularly have more classes of the same name.

Second: Use google guava cache with weak keys and values.

Third: If your cache is loaded by the application class loader, there should be no problem at all....

Fourth: If you can determine the number of classloaders (=applications) in your server using your service and the number of classes to be cached, go with a standard cache and just adjust the size of the elements it holds. The cache will remove unused entries by the eviction policy.