CODEWITHSUNDEEP
maven-pluginapache-nifi
Mike
Mike

Reputation: 861

Can I use another build system to build custom processors with NiFi?

My company uses SBT/Scala so my goal is to develop our custom NiFi processors with these rather than Maven/Java. The problem I'm having is I don't know how to generate the NAR files with SBT. NiFi has a nifi-nar-maven-plugin package plugin that it uses to package all the processor dependencies into a jar (nar). Is there anyway to build the NAR using SBT?

Options I've considered:

Upvotes: 0

Views: 1070

Answers (2)

tonykoval
tonykoval

Reputation: 1257

I created a SBT plugin that is inspired from sbt-pack and nifi-maven.

Upvotes: 0

Bryan Bende
Bryan Bende

Reputation: 18670

NARs are used to provided class-loader isolation in NiFi. Each NAR is an artifact that contains all of the JARs it needs, and they are only visible to that NAR. This prevents against conflicting libraries used by different NARs, so if one NAR uses Guava 16 and another NAR uses Guava 18, it won't cause a problem.

NARs get unpacked to nifi_home/work/nar. For example, looking at nifi-0.7.0/work/nar/extensions/nifi-ambari-nar-0.7.0.nar-unpacked/META-INF/bundled-dependencies/ it would show all the JARS that were included for the ambari NAR.

Each NAR can have a dependency on one other NAR, this is how processors can have a NAR dependency on controller services. In these cases, the NAR Maven plugin creates a special MANIFEST file in the NAR that specifies the id of a parent NAR, and that id is used by NiFi to perform the appropriate class loading.

The developer guide covers some of this: https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars

Personally I would recommend using Maven since the NAR plugin is the standard way NARs are produced. As you pointed out it is probably possible to create a similar plugin with SBT, but seems like a lot of effort. Using Maven doesn't mean you can't use Scala... This project offers a template for developing processors in Scala, but still uses Maven as the build tool:

https://github.com/jfrazee/nifi-processor-bundle-scala.g8

Upvotes: 4

Related Questions