Get declared fields of java.lang.reflect.Fields in jdk12

Question

In java8 it was possible to access fields of class java.lang.reflect.Fields using e.g.

Field.class.getDeclaredFields();

In java12 (starting with java9 ?) this returns only a empty array. This doesn't change even with

--add-opens java.base/java.lang.reflect=ALL-UNNAMED

set.

Any ideas how to achieve this? (Appart from the fact that this might be a bad idea, i want to be able to change a "static final" field in my code during junit testing via reflection. This has been possible with java8 by changing the "modifiers"

Field modifiersField = Field.class.getDeclaredField("modifiers");
modifiersField.setAccessible(true);
modifiersField.setInt(myfield, myfield.getModifiers() & ~Modifier.FINAL);

)

Answer 1

It is possible without java.misc.Unsafe on Java 21 with the following Code and JVM parameters:

JVM-Parameters:

--add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-exports=java.base/java.lang.invoke=ALL-UNNAMED --add-exports=java.base/jdk.internal.access=ALL-UNNAMED --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED

Java-Code

public class HiddenFieldsRevealed {

    private static final Object greeting = "Hello world";

    public static void main(String... args) throws Throwable {

        Field myStaticFinalField = HiddenFieldsRevealed.class.getDeclaredField("greeting");
        myStaticFinalField.setAccessible(true);

        removeFinalness(myStaticFinalField);

        //Logic taken from java.lang.invoke.MethodHandle.unreflectSetter(Field)
        
        //.unreflectSetter(Field) 
        //=> .unreflectField(Field, false)
        //=> lookup.getDirectFieldNoSecurityManager(memberName.getReferenceKind(), f.getDeclaringClass(), memberName);
        //=> .getDirectFieldCommon(refKind, referenceClass, memberName, false)
        
        Class<?> memberNameClass = Class.forName("java.lang.invoke.MemberName");
        
        Constructor<?> memberNameConstructor = memberNameClass.getDeclaredConstructor(Field.class, boolean.class);
        memberNameConstructor.setAccessible(true);

        Object memberNameInstanceForField = memberNameConstructor.newInstance(myStaticFinalField, true);
        
        Field memberNameFlagsField = memberNameClass.getDeclaredField("flags");
        
        memberNameFlagsField.setAccessible(true);
        
        //Manipulate flags to remove hints to it being final
        memberNameFlagsField.setInt(memberNameInstanceForField, (int)memberNameFlagsField.getInt(memberNameInstanceForField) & ~Modifier.FINAL);
        
        Method getReferenceKindMethod = memberNameClass.getDeclaredMethod("getReferenceKind");
        
        getReferenceKindMethod.setAccessible(true);
        
        byte getReferenceKind = (byte)getReferenceKindMethod.invoke(memberNameInstanceForField);

        MethodHandles.Lookup mh = MethodHandles.privateLookupIn(HiddenFieldsRevealed.class, MethodHandles.lookup());

        Method getDirectFieldCommonMethod = mh.getClass().getDeclaredMethod("getDirectFieldCommon", byte.class, Class.class, memberNameClass, boolean.class);
        
        getDirectFieldCommonMethod.setAccessible(true);
        
        //Invoke last method to obtain the method handle

        MethodHandle o = (MethodHandle) getDirectFieldCommonMethod.invoke(mh, getReferenceKind, myStaticFinalField.getDeclaringClass(), memberNameInstanceForField, false);
        
        o.invoke("ModifiedValue");

        System.out.println(greeting);

    }

    public static void removeFinalness(Field field) throws Throwable {
        Method[] classMethods = Class.class.getDeclaredMethods();

        Method declaredFieldMethod = Arrays.stream(classMethods).filter(x -> Objects.equals(x.getName(), "getDeclaredFields0")).findAny().orElseThrow();

        declaredFieldMethod.setAccessible(true);

        Field[] declaredFieldsOfField = (Field[]) declaredFieldMethod.invoke(Field.class, false);

        Field modifiersField = Arrays.stream(declaredFieldsOfField).filter(x -> Objects.equals(x.getName(), "modifiers")).findAny().orElseThrow();

        modifiersField.setAccessible(true);

        modifiersField.setInt(field, field.getModifiers() & ~Modifier.FINAL);

    }
}

As of Java 21 this works because of the direct call to getDirectFieldCommon(...) that we can fool by modifying the flags on MemberName instance for a field.

Considering that Unsafe.getStaticFieldOffset(...) is deprecated and this workaround is prone to break, I hope that there will be always a way in the future to modify static final fields at runtime because there are some nasty dependencies that are easier to deal with this way than to have the whole ordeal of writing something that instruments the class or to patch the library on compile with a build system just to remove a final where it should not have been in the first place.

Answer 2

Why It No Longer Works

The reason this no longer works in Java 12 is due to JDK-8210522. This CSR says:

Summary

Core reflection has a filtering mechanism to hide security and integrity sensitive fields and methods from Class getXXXField(s) and getXXXMethod(s). The filtering mechanism has been used for several releases to hide security sensitive fields such as System.security and Class.classLoader.

This CSR proposes to extend the filters to hide fields from a number of highly security sensitive classes in java.lang.reflect and java.lang.invoke.

Problem

Many of classes in java.lang.reflect and java.lang.invoke packages have private fields that, if accessed directly, will compromise the runtime or crash the VM. Ideally all non-public/non-protected fields of classes in java.base would be filtered by core reflection and not be readable/writable via the Unsafe API but we are no where near this at this time. In the mean-time the filtering mechanism is used as a band aid.

Solution

Extend the filter to all fields in the following classes:

java.lang.ClassLoader
java.lang.reflect.AccessibleObject
java.lang.reflect.Constructor
java.lang.reflect.Field
java.lang.reflect.Method

and the private fields in java.lang.invoke.MethodHandles.Lookup that are used for the lookup class and access mode.

Specification

There are no specification changes, this is filtering of non-public/non-protected fields that nothing outside of java.base should rely on. None of the classes are serializable.

Basically, they filter out the fields of java.lang.reflect.Field so you can't abuse them—as you're currently trying to do. You should find another way to do what you need; the answer by Eugene appears to provide at least one option.

---

Proper Fix

The proper way to drop a final modifier is to instrument the running program, and have your agent redefine the class. If you do this when the class is first loaded, it's no different than having modified the class file before the JVM was even started. In other words, it's like the final modifier was never present.

---

Workaround

Obligatory Warning: The developers of Java obviously don't want you to be able to change a final field into a non-final field without actually changing the class file (e.g., by recompiling the source code, instrumentation, etc.). Use any hack at your own risk; it may have unintended side-effects, work only some times, and/or stop working in a future release.

Use java.lang.invoke

The following uses the java.lang.invoke package. For whatever reason, the same restrictions applied to the Reflection API are not applied to the Invoke API (at least up to and including Java 17; continue reading for more information).

The example modifies the EMPTY_ELEMENTDATA final field of the ArrayList class. This field normally contains an empty array that's shared between all ArrayList instances when initialized with a capacity of 0. The below sets the field to {"Hello", "World!"}, and as you can see by running the program, this results in the list instance containing elements that were never added to it.

Java 12 - 17

I tested this on Java 16.0.2 and Java 17.0.3, both downloaded from https://adoptium.net/.

import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayList;

public class Main {

  private static final VarHandle MODIFIERS;

  static {
    try {
      var lookup = MethodHandles.privateLookupIn(Field.class, MethodHandles.lookup());
      MODIFIERS = lookup.findVarHandle(Field.class, "modifiers", int.class);
    } catch (IllegalAccessException | NoSuchFieldException ex) {
      throw new RuntimeException(ex);
    }
  }

  public static void main(String[] args) throws Exception {
    var emptyElementDataField = ArrayList.class.getDeclaredField("EMPTY_ELEMENTDATA");
    // make field non-final
    MODIFIERS.set(emptyElementDataField, emptyElementDataField.getModifiers() & ~Modifier.FINAL);
    
    // set field to new value
    emptyElementDataField.setAccessible(true);
    emptyElementDataField.set(null, new Object[] {"Hello", "World!"});

    var list = new ArrayList<>(0);

    // println uses toString(), and ArrayList.toString() indirectly relies on 'size'
    var sizeField = ArrayList.class.getDeclaredField("size");
    sizeField.setAccessible(true);
    sizeField.set(list, 2); // the new "empty element data" has a length of 2

    System.out.println(list);
  }
}

Run the code with:

javac Main.java
java --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED Main

^{Note: I tried to use the "single source file" feature, but that resulted in a ConcurrentModificationException. As pointed out in the comments, this is likely due to some JIT optimization (e.g., the static final field has been inlined, because the JVM does not expect such a field to be able to change).}

Output:

[Hello, World!]

Java 18+

Unfortunately, the above results in the following exception on Java 18.0.1 (downloaded from https://adoptium.net/):

Exception in thread "main" java.lang.UnsupportedOperationException
        at java.base/java.lang.invoke.VarForm.getMemberName(VarForm.java:114)
        at Main.main(Main.java:23)

Where line 23 is:

MODIFIERS.set(emptyElementDataField, emptyElementDataField.getModifiers() & ~Modifier.FINAL);

Answer 3

This works in JDK 17.

import java.lang.reflect.Field;
import sun.misc.Unsafe;

/**
 * @author Arnah
 * @since Feb 21, 2021
 **/
public class FieldUtil{

    private static Unsafe unsafe;

    static{
        try{
            final Field unsafeField = Unsafe.class.getDeclaredField("theUnsafe");
            unsafeField.setAccessible(true);
            unsafe = (Unsafe) unsafeField.get(null);
        }catch(Exception ex){
            ex.printStackTrace();
        }
    }

    public static void setFinalStatic(Field field, Object value) throws Exception{
        Object fieldBase = unsafe.staticFieldBase(field);
        long fieldOffset = unsafe.staticFieldOffset(field);

        unsafe.putObject(fieldBase, fieldOffset, value);
    }
}

public class YourClass{
    public static final int MAX_ITEM_ROWS = 35_000;
}

FieldUtil.setFinalStatic(YourClass.class.getDeclaredField("MAX_ITEM_ROWS"), 1);

Answer 4

I found a way and it worked on JDK 8, 11, 17.

Method getDeclaredFields0 = Class.class.getDeclaredMethod("getDeclaredFields0", boolean.class);
getDeclaredFields0.setAccessible(true);
Field[] fields = (Field[]) getDeclaredFields0.invoke(Field.class, false);
Field modifiers = null;
for (Field each : fields) {
    if ("modifiers".equals(each.getName())) {
        modifiers = each;
        break;
    }
}
assertNotNull(modifiers);

Don't forget to set the following args when using JDK 11 or higher:

--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED

Answer 5

You can't. This was a change done on purpose.

For example, you could use PowerMock and it's @PrepareForTest - under the hood it uses javassist (bytecode manipulation) if you want to use that for testing purposes. This is exactly what that bug in the comments suggests to do.

In other words, since java-12 - there is no way to access that via vanilla java.