Apache NiFi Custom NAR NoClassDefFoundError During Startup

Question

I am attempting to install a custom NAR controller service bundle in my NiFi installation but get this error during startup. It acts like it can't find the RecordReaderFactory class from the nifi-record-serialization-service-api even though it is included as a dependency:

<dependency>
   <groupId>org.apache.nifi</groupId>
   <artifactId>nifi-record-serialization-service-api</artifactId>
   <version>1.9.0</version>
</dependency>

Here is the project: https://github.com/adamfisher/nifi-zonefile-record-serialization-service

Running maven install builds the NAR successfully. It's just when I startup NiFi, it gives this error. I'm pretty sure this is a maven POM configuration issue. I just don't work with Java too much and hoping someone can shed some light on why it is not finding the dependent class it needs?

nifi-app.log:

2019-03-02 15:22:15,245 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: D:\nifi\NIFI-1~2.0\.\work\nar\framework\nifi-framework-nar-1.9.0.nar-unpacked\NAR-INF\bundled-dependencies\nifi-web-error-1.9.0.war with context path set to /
2019-03-02 15:22:15,261 INFO [main] org.apache.nifi.web.server.JettyServer Running in HTTP mode; host headers not restricted
2019-03-02 15:22:16,386 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.NoClassDefFoundError: org/apache/nifi/serialization/RecordReaderFactory
java.lang.NoClassDefFoundError: org/apache/nifi/serialization/RecordReaderFactory
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at org.apache.nifi.nar.StandardExtensionDiscoveringManager.loadExtensions(StandardExtensionDiscoveringManager.java:152)
    at org.apache.nifi.nar.StandardExtensionDiscoveringManager.discoverExtensions(StandardExtensionDiscoveringManager.java:127)
    at org.apache.nifi.nar.StandardExtensionDiscoveringManager.discoverExtensions(StandardExtensionDiscoveringManager.java:113)
    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:925)
    at org.apache.nifi.NiFi.<init>(NiFi.java:158)
    at org.apache.nifi.NiFi.<init>(NiFi.java:72)
    at org.apache.nifi.NiFi.main(NiFi.java:297)
Caused by: java.lang.ClassNotFoundException: org.apache.nifi.serialization.RecordReaderFactory
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 23 common frames omitted
2019-03-02 15:22:16,387 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2019-03-02 15:22:16,387 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).

Answer 1

I have the same issue as described here and I managed to solved it including the dependencies at the nar module(nifi-base-nar) pom.xml

The dependency that you need to include is the following:

<dependency>
    <groupId>org.apache.nifi</groupId>
    <artifactId>nifi-standard-services-api-nar</artifactId>
    <version>${nifi.version}</version>
    <type>nar</type>
</dependency>

Answer 2

I have spent a lot of time on this. Finally, i have a working solution for me. My custom processor is using RecordReader for incoming stream. However, either compile or package failed in maven process. I had to customize pom.xml as following:

<dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-record-serialization-service-api</artifactId>
        <version>${nifi.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-record-serialization-services-nar</artifactId>
        <version>${nifi.version}</version>
        <type>nar</type>
    </dependency>

Answer 3

The NAR dependencies are used at runtime to create a chain of parent class loaders. So in your example you have

nifi-zonefile-record-serialization-service-nar -> (depends on)

nifi-zonefile-record-serialization-service-api-nar -> (depends on)

nifi-standard-services-api-nar

In the running application when it instantiates the instance of your record reader from your service NAR, it then needs to load the interface that implements which comes from your service API NAR, which then needs to load the RecordReader interface which comes from the standard services API NAR.

Answer 4

I ended up needing to include the API NAR file as well even though it doesn't appear to be used anywhere. It's just a generic template class called MyService.