(de)serialization of parsed Groovy scripts

Question

We want to enable our customers to customize certain aspects of their requests processing, by letting them write something (currently looking at Groovy scripts), then have those scripts saved in a DB and applied when necessary, this way we won't have to maintain all those tiny aspects of processing details that might apply to certain customers only.

So, with Groovy, a naive implementation would go like this:

1. GroovyShell shell = new GroovyShell(); // prepare execution engine - probably once per thread
2. (retrieve script body from the DB, when necessary)
3. Script script = shell.parse(scriptBody); // parse/compile execution unit
4. Binding binding = prepareBinding(..); script.setBinding(binding); // provide script instance with execution context
5. script.run(); doSomething(binding);

When run one after the other, step 1 takes approx. 800 msec, step 3 takes almost 2000 msec, and step 5 takes about 150 msec. Absolute numbers will vary, but the relative numbers are quite stable. Assuming that step 1 is not going to be executed per-request, and step 5 number execution time is quite tolerable, I am very much concerned with step 3: parsing the Groovy script instance from the source code. I did some reading across the documentation and code, and some googling as well, but has not thus far discovered any solution, so here's the question:

Can we somehow pre-compile groovy code ONCE, then persist it in the DB and then re-hydrate whenever necessary, to obtain an executable Script instance (that we could also cache when necessary) ?

Or (as I am just thinking now) we could just compile Java code to bytecode and persist it in the Db?? Anyway, I am not so much concerned about particular language used for the scripts, but sub-second execution time is a must.. Thanks for any hints!

NB: I am aware that GroovyShellEngine will likely cache the compiled script; that still risks too long of a delay for first time execution, also risks memory overconsumption...

UPD1: based on excellent suggestion by @daggett, I've modified a solution to look as follows:

GroovyShell shell = new GroovyShell();
final Class<? extends MetaClass> theClass = shell.parse(scriptBody).getMetaClass().getTheClass();

Script script = InvokerHelper.createScript(theClass, binding);
script.run();

this works all fine and well! Now, we need to de-couple metaclass creation and usage; for that, I've created a helper method:

    private Class dehydrateClass(Class theClass) throws IOException, ClassNotFoundException {
        final ByteArrayOutputStream stream = new ByteArrayOutputStream();
        ObjectOutputStream outputStream = new ObjectOutputStream(stream);
        outputStream.writeObject(theClass);
        InputStream in = new ByteArrayInputStream(stream.toByteArray());
        final ObjectInputStream inputStream = new ObjectInputStream(in);
        return (Class) inputStream.readObject();
    }

which I've dested as follows:

    @Test
    void testDehydratedClass() throws IOException, ClassNotFoundException, IllegalAccessException, InstantiationException {
        RandomClass instance = (RandomClass) dehydrateClass(RandomClass.class).newInstance();
        assertThat(instance.getName()).isEqualTo("Test");
    }

    public static class RandomClass {
        private final String name;

        public RandomClass() {
            this("Test");
        }

        public RandomClass(String name) {
            this.name = name;
        }

        public String getName() {
            return this.name;
        }
    }

which passes OK, which means that, in general, this approach is OK.

However, when I try to apply this dehydrateClass approach to theClass, returned by compile phase, I get this exception:

java.lang.ClassNotFoundException: Script1

    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:686)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749)
    at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1714)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1554)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)

so, my impression is, that this de-serialization trick will not do any good, if the ClassLoader in question does not already have knowledge of what constitutes a Script1.. seems like the only way to make this kind of approach work is to save those pre-compiled classes somehow somewhere .. or may be learn to serialize them differently

Answer 1

you can parse/compile scripts/classes during editing and store compiled version somewhere - in database, file system, memory, ...

here is a groovy code snippet to compile script/class to a bytecode and then define/load classes from the bytecode.

import org.codehaus.groovy.control.BytecodeProcessor
import org.codehaus.groovy.control.CompilerConfiguration

//bytecode processor that could be used to store bytecode to cache(file,db,...)
@groovy.transform.CompileStatic
class BCP implements BytecodeProcessor{
    Map<String,byte[]> bytecodeMap = [:]
    byte[] processBytecode(String name, byte[] original){
        println "$name >> ${original.length}"
        bytecodeMap[name]=original //here we could store bytecode to a database or file system instead of memory map...
        return original
    }
}

def bcp = new BCP()
//------ COMPILE PHASE
def cc1 = new CompilerConfiguration()
cc1.setBytecodePostprocessor(bcp)
def gs1 = new GroovyShell(new GroovyClassLoader(), cc1)
//the next line will define 2 classes: MyConst and MyAdd (extends Script) named after the filename
gs1.parse("class MyConst{static int cnt=0} \n x+y+(++MyConst.cnt)", "MyAdd.groovy")

//------ RUN PHASE
//   let's create another classloader that has no information about classes MyAdd and MyConst 
def cl2 = new GroovyClassLoader()

//this try-catch just to test that MyAdd fails to load at this point 
// because unknown for 2-nd class loader
try {
    cl2.loadClass("MyAdd")
    assert 1==0: "this should not happen because previous line should throw exception"
}catch(ClassNotFoundException e){}

//now define previously compiled classes from the bytecode
//you can load bytecode from filesystem or from database
//for test purpose let's take them from map
bcp.bytecodeMap.each{String name, byte[] bytes->
    cl2.defineClass(name, bytes)
}

def myAdd = cl2.loadClass("MyAdd").newInstance()
assert myAdd instanceof groovy.lang.Script //it's a script

myAdd.setBinding([x: 1000, y: 2000] as Binding)
assert myAdd.run() == 3001 // +1 because we have x+y+(++MyConst.cnt)

myAdd.setBinding([x: 1100, y: 2200] as Binding)
assert myAdd.run() == 3302 

println "OK"