CODEWITHSUNDEEP
apache-nifi
ab m
ab m

Reputation: 422

NiFi fetchFile processor doesn't allow dynamic attributes

What is the reason that some of NiFi processors don't allow dynamic attributes? I'm using FetchFile processor in one of my workflows and I need to pass through some data throughout the flow to be able to use it in the last step. However, FetchFile breaks it by not allowing dynamic attributes. I'm wondering if there is another way to do it? Why would NiFi not allow dynamic attributes on certain processors?

My flow is something like

ExecuteScript -> EvaluateJSon -> Custom Processor to write files-> FetchFile->SendtoS3 -> Mark workflow complete

I want to send some metadata so that I could mark the workflow complete. I'm passing that data as attributes but it breaks at FetchFile.

Upvotes: 2

Views: 2032

Answers (1)

Bryan Bende
Bryan Bende

Reputation: 18670

There are two separate concepts, user-defined properties on processors, and flow file attributes.

User-defined properties let a processor take input from a user for something that couldn't be defined ahead of time. Examples of this are in EvaluateJsonPath when the JSON paths are specified in user-defined properties, or in PutSolrContentStream when all the user-defined properties get passed as query parameters to Solr.

FlowFile attributes are a map of key/value pairs that get passed around with each piece of data. These attributes are usually created when a processor produces or modifies a flow file, or can be manipulated using processors like UpdateAttribute.

It is up to each processor to decide whether it needs user-defined properties and how they would be used. UpdateAttribute happens to be a processor where the user-defined properties are added as new key/value pairs to each flow file, but it doesn't make sense for every processor to do that.

Upvotes: 5

Related Questions