CODEWITHSUNDEEP
hazelcasthazelcast-imap
Ben
Ben

Reputation: 6887

Hazelcast: LinkageError "attempted duplicate class definition for name"

I saw this error in our hazelcast cluster.

The code is trying to call executeOnKey() on an IMap. e.g.

    IMap<MyKey, MyCachedClass> myMap = hz.getMap("my-map");
    myMap.executeOnKey(myKey, new EntryProcessor() {
        @Override
        public Object process(Map.Entry entry) {
            return entry.getValue().setItem(item);
        }

        @Override
        public EntryBackupProcessor getBackupProcessor() {
            return null;
        }
    });

...and am getting this exception:

Exception in thread "MYTHREAD" java.lang.LinkageError: loader (instance of com/hazelcast/internal/usercodedeployment/impl/ClassSource): attempted duplicate class definition for name: "com/mycompany/model/Item$ItemEnum"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
at com.hazelcast.internal.usercodedeployment.impl.ClassSource.define(ClassSource.java:50)
at com.hazelcast.internal.usercodedeployment.impl.ClassLocator.tryToGetClassFromRemote(ClassLocator.java:163)
at com.hazelcast.internal.usercodedeployment.impl.ClassLocator.handleClassNotFoundException(ClassLocator.java:95)
at com.hazelcast.internal.usercodedeployment.impl.ClassSource.loadClass(ClassSource.java:65)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
at com.hazelcast.internal.usercodedeployment.impl.ClassSource.define(ClassSource.java:50)
at com.hazelcast.internal.usercodedeployment.impl.ClassLocator.tryToGetClassFromRemote(ClassLocator.java:161)
at com.hazelcast.internal.usercodedeployment.impl.ClassLocator.handleClassNotFoundException(ClassLocator.java:95)
at com.hazelcast.internal.usercodedeployment.UserCodeDeploymentService.handleClassNotFoundException(UserCodeDeploymentService.java:89)
at com.hazelcast.internal.usercodedeployment.UserCodeDeploymentClassLoader.loadClass(UserCodeDeploymentClassLoader.java:57)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:288)
at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:237)
at com.hazelcast.nio.IOUtil$ClassLoaderAwareObjectInputStream.resolveClass(IOUtil.java:646)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at java.util.ArrayList.readObject(ArrayList.java:797)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:82)
at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:75)
at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:269)
at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:574)
at com.hazelcast.map.impl.operation.EntryOperation.readInternal(EntryOperation.java:263)
at com.hazelcast.spi.Operation.readData(Operation.java:728)
at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187)
at com.hazelcast.spi.impl.NodeEngineImpl.toObject(NodeEngineImpl.java:323)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:398)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:153)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:123)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:110)
at ------ submitted from ------.(Unknown Source)
at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolve(InvocationFuture.java:127)
at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveAndThrowIfException(InvocationFuture.java:79)
at com.hazelcast.spi.impl.AbstractInvocationFuture.get(AbstractInvocationFuture.java:162)
at com.hazelcast.map.impl.proxy.MapProxySupport.executeOnKeyInternal(MapProxySupport.java:1099)
at com.hazelcast.map.impl.proxy.MapProxyImpl.executeOnKeyInternal(MapProxyImpl.java:109)
at com.hazelcast.map.impl.proxy.MapProxyImpl.executeOnKey(MapProxyImpl.java:757)
at com.mycompany.Processor.java

So to me this seemed like:

I just cannot understand why it's getting a duplicate class exception, when it first claimed it couldn't find the class. And this error is proving difficult to replicate.

We do have 'userCodeDeployment' set to allow loading of remote class definitions. Our config looks like this:

UserCodeDeploymentConfig ucdConfig = new UserCodeDeploymentConfig();
ucdConfig.setEnabled(true);
ucdConfig.setClassCacheMode(UserCodeDeploymentConfig.ClassCacheMode.ETERNAL);
ucdConfig.setProviderMode(UserCodeDeploymentConfig.ProviderMode.LOCAL_AND_CACHED_CLASSES);
config.setUserCodeDeploymentConfig(ucdConfig);

So it sort of looks like it's doing what I'd expect. i.e. getting a remote class definition because it can't find it locally. Is just a puzzle as to why the exception is occurring.

Upvotes: 0

Views: 2747

Answers (2)

Ben
Ben

Reputation: 6887

Eventually worked this one out.

A simple example.

  • Assume two Hazelcast nodes. Both have userCodeDeployment switched on. (The ability to share class definitions over the wire.)
  • Node 1 has MyClass on it's classpath, Node 2 does not.
  • MyClass definition, note the inner enum (I think the inner enum can be swapped for an inner class and the same thing will happen).
public class MyClass implements Serializable {

    private final InnerEnum enumVal;

    public MyClass(InnerEnum enumVal) {
        this.enumVal = enumVal;
    }

    public enum InnerEnum {
        ONE, TWO, THREE;
    }
}

From Node 1, the following code is run:

hz.getMap("myClassMap").put("myClassKey", new MyClass(MyClass.InnerEnum.ONE));
hz.getMap("myInnerEnumMap").put("myInnerEnumKey", MyClass.InnerEnum.TWO);

Then, on Node 2, the following code is run:

System.out.println(hz.getMap("myInnerEnumMap").get("myInnerEnumKey"));
System.out.println(hz.getMap("myClassMap").get("myClassKey"));

On Node 2, I see the following logged:

[TRACE] ClassLocator Loaded class com.mycompany.MyClass$InnerEnum from Member [myhost]:5701 - cc0b2872-7f5b-484b-a40f-7c7ba1fdc165
TWO
[TRACE] ClassLocator Loaded class com.mycompany.MyClass from Member [myhost]:5701 - cc0b2872-7f5b-484b-a40f-7c7ba1fdc165

Exception in thread "main" java.lang.LinkageError: loader (instance of  com/hazelcast/internal/usercodedeployment/impl/ClassSource): attempted  duplicate class definition for name: "com/mycompany/MyClass$InnerEnum"
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
    at com.hazelcast.internal.usercodedeployment.impl.ClassSource.define(ClassSource.java:50)
    at com.hazelcast.internal.usercodedeployment.impl.ClassLocator.tryToGetClassFromRemote(ClassLocator.java:163)
    at com.hazelcast.internal.usercodedeployment.impl.ClassLocator.handleClassNotFoundException(ClassLocator.java:95)
    at com.hazelcast.internal.usercodedeployment.UserCodeDeploymentService.handleClassNotFoundException(UserCodeDeploymentService.java:89)
    at com.hazelcast.internal.usercodedeployment.UserCodeDeploymentClassLoader.loadClass(UserCodeDeploymentClassLoader.java:57)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:288)
    at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:237)
    at com.hazelcast.nio.IOUtil$ClassLoaderAwareObjectInputStream.resolveClass(IOUtil.java:646)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
    at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:82)
    at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:75)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187)
    at com.hazelcast.map.impl.proxy.MapProxySupport.toObject(MapProxySupport.java:1245)
    at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:120)
    at com.mycompany.main(Node2.java:74)

So:

  1. Node 2 is first trying to fetch and display an instance of MyClass$InnerEnum. It fetches the InnerEnum class definition from Node 1 over hazelcast.
  2. Node 2 then tries to fetch and display an instance of MyClass. It fetches the MyClass class definition from Node 1.
  3. Hazelcast then loops through the inner classes of MyClass (which just contains InnerEnum) and asks the class loader to define them without checking whether they already exist.

See (from Hazelcast 3.11.1) com.hazelcast.internal.usercodedeployment.impl.ClassLocator:160

Map<String, byte[]> innerClassDefinitions = classData.getInnerClassDefinitions();
classSource.define(name, classData.getMainClassDefinition());
for (Map.Entry<String, byte[]> entry : innerClassDefinitions.entrySet()) {
    classSource.define(entry.getKey(), entry.getValue());
}

The easiest solution in our case was to pull InnerEnum out as a top level Enum.

Upvotes: 1

wildnez
wildnez

Reputation: 1098

Try creating the EntryProcessor as a static or a top level class and pass the item explicitly in its constructor. That hopefully should resolve the issue.

The JVM can get a little tricky with dynamic class loading, which is what is done by UserCodeDeployment.

Upvotes: 0

Related Questions