CODEWITHSUNDEEP
javajava-bytecode-asmjavaagentsjvm-bytecode
starflyer
starflyer

Reputation: 485

Stack overflow error in java.lang.instrument with ASM bytecode transformation

I'm new to Java agent instrumentation and ASM bytecode instrumentation. I took the code from this UCLA tutorial and used it for javagent instrumentation using java.lang.instrument.

First question, is there anything in ASM bytecode library that is incompatible with javaagent instrumentation?

Here's the program in a somewhat redacted form:

public class Instrumenter {
    public static void premain(String args, Instrumentation inst) throws Exception {
        Transformer tr = new Transformer();
        inst.addTransformer(tr);
    }

}

class Transformer implements ClassFileTransformer {

    public Transformer() {
    }

    @Override
    public byte[] transform( ClassLoader loader, String className, Class<?> klass, ProtectionDomain domain, byte[] klassFileBuffer ) throws IllegalClassFormatException {
        byte[] barray;
        ClassWriter cwriter = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
        ClassReader creader;
        try {
            creader = new ClassReader(new ByteArrayInputStream(klassFileBuffer));
        } catch (Exception exc) {
            throw new IllegalClassFormatException(exc.getMessage());
        }
        ClassVisitor cvisitor = new ClassAdapter(cwriter);
        creader.accept(cvisitor, 0);
        barray = cwriter.toByteArray();
        return barray;
    }

}

class ClassAdapter extends ClassVisitor implements Opcodes {
    public ClassAdapter(ClassVisitor cv) {
        super(ASM7, cv);
    }


    @Override
    public MethodVisitor visitMethod( final int access, final String name, final String desc, final String signature, final String[] exceptions ) {
        this.pwriter.println(ClassAdapter.nextMethodId + "," + this.className + "#" + name);
        MethodVisitor mv = cv.visitMethod(access, name, desc, signature, exceptions);
        if (mv == null) {
            return null;
        } else {
            return new MethodAdapter(mv);
        }
    }
}

class MethodAdapter extends MethodVisitor implements Opcodes {
    public MethodAdapter(final MethodVisitor mv) {
        super(ASM7, mv);
    }

    @Override
    public void visitMethodInsn(int opcode, String owner, String name, String desc, boolean itf) {
        mv.visitFieldInsn(GETSTATIC, "java/lang/System", "err", "Ljava/io/PrintStream;");
        mv.visitLdcInsn("CALL " + name);
        mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false);
        // do call
        mv.visitMethodInsn(opcode, owner, name, desc, itf);
        mv.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "err", "Ljava/io/PrintStream;");
        mv.visitLdcInsn("RETURN " + name);
        mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false);
    }
}

So the javaagent instrumentation works on small programs. I tried running it on the DaCapo benchmark suite and it throws a StackOverflowError like so:

Exception: java.lang.StackOverflowError thrown from the UncaughtExceptionHandler in thread "main"

When I remove the instructions added in visitMethodInsn, the agent runs successfully. I researched this bit some more and found something in the ASM docs about having to call MethodVisitor.visitMaxs. This seems to be the most likely reason for the StackOverflowError.

So further questions:

Upvotes: 2

Views: 920

Answers (1)

Holger
Holger

Reputation: 298143

When you register a ClassFileTransformer, it will be invoked for every subsequently loaded class. This may include classes used by the print operation itself you are injecting, if these classes have not been used before. You are injecting print statements for every method invocation, including constructor invocations, and the operations behind System.err.println(…) will involve method invocations and object constructions, so if these got instrumented, they will enter another printing operation and this recursion will lead to a StackOverflowError.

Apparently, an UncaughtExceptionHandler is installed, which tries to print the StackOverflowError, which, being itself instrumented the same way, will lead again to a StackOverflowError, so the error message reads like “StackOverflowError thrown from the UncaughtExceptionHandler”.

You should restrict, which classes you are instrumenting. E.g. you may not transform the class when its loader is null, to exclude all classes loaded by the bootstrap class loader. Or you check the name argument to exclude classes starting with java.. Or more elaborated solution would be to enhance the code you’re injecting, to detect when it is within an injected print operation and not go into recursion.

By the way, use new ClassReader(klassFileBuffer) and you don’t need a try … catch block. Further, when you insert code as simple as yours, you may use ClassWriter.COMPUTE_MAXS instead of ClassWriter.COMPUTE_FRAMES, to avoid expensive recalculations of the stack map frames. Since you don’t specify SKIP_FRAMES to the reader, it will report the original frames to the writer and ASM is capable of adapting the positions, so it’s no problem when you insert some simple instructions. Only when you insert or remove branches or introduce variables which have to persist across branches, you need to adapt or recalculate the frames.

Upvotes: 3

Related Questions