CODEWITHSUNDEEP
javaapache-nifi
Lasitha Yapa
Lasitha Yapa

Reputation: 4609

Update attributes based on content in NiFi

How can I add a attribute to the current flow file when developing an Apache NiFi cusom processor.
What I want to do is adding a new attribute (or at least update a current attribute) to the current flow file with calculated value.
Or is there an already built processor that I can use?

Upvotes: 6

Views: 24327

Answers (2)

Paul Jackson
Paul Jackson

Reputation: 2147

I had a use case where I needed to load many attributes from a Java-style Properties file. This could be done with ExtractText but it required adding a property and regular expression for every property to be supported. I thought it would be nice to support whatever properties were in the file without having to configure the processor for each one.

The solution I came up with was to use the ExecuteGroovyScript processor with the following script:

def ff=session.get();
if (ff != null) {
  def properties = new Properties();
  def is = ff.read();
  properties.load(is);
  is.close();
  ff.putAllAttributes(properties);
  REL_SUCCESS << ff;
}

This script reads the properties from a physical file rather than from the flow file.

def ff=session.get();
if (ff != null) {
  def properties = new Properties();
  def propertiesFile = new File('/Users/me/mydata/foo.properties')
  propertiesFile.withInputStream {
    properties.load(it)
  }
  ff.putAllAttributes(properties);
  REL_SUCCESS << ff;
}

Upvotes: 0

James
James

Reputation: 11931

NiFi supports several methods of creating and updating attributes, depending on the data source you wish to use. Some general purpose processors include:

UpdateAttribute - Updates attributes on flow files using both static values and NiFi's expression language.

enter image description here

You can add as many properties with one processor. I recommend scanning through the Apache NiFi Expression Language Guide to get a feel for what you can do with it.

ExtractText - The Sets attribute values by applying regular expressions to the flowfile content.

ExecuteScript - ExecuteScript Runs custom script code, which can be used to update attributes however you wish.

And there are more for particular content formats, for example:

Upvotes: 15

Related Questions