CODEWITHSUNDEEP
apache-nifi
Rao
Rao

Reputation: 892

How to load value from dynamically specified parameter in NiFi

I have several processes with almost same flow like "Get some parameters, extract data from database according to them and upload them to target". The parameters vary slightly across processes as well as targets but only a bit. Most of the process is the same. I would like to extract those differences to parameter-context and dynamically load them. My idea is to have parameters defined following way and then using them.

Groups of parameters

So core of question is:

How to dynamically choose which parameter group load and use?

Having several parameter contexts with same-named/different-valued parameters and dynamically switching them would be probably the best, but it is not possible as far as I know.

Also duplicating flows is out-of-the-table. Any error correction would be spread out over several places and maintenance would be a nightmare.

Moreover, I know I can do it like "In GenetrateFlowFile for process A set value1=#{A_value1} and in GenetrateFlowFile for process B set value1=#{B_value1}. But this is tedious, error-prone and scales kinda bad. Not speaking of situation when I can have dozens of parameters and several processes. Also it is a kind of hardcoding, not configuring...

I was hoping for something like defining group=A and then using it like value1=#{ ${ group:append('_value1') } } but this does not work - it is evaluated as parameter literally named ${ group:append('_value1') }.

Upvotes: 0

Views: 2114

Answers (1)

Rao
Rao

Reputation: 892

TL;DR: Use evaluateELString().

The actual solution is to set in GenetrateFlowFile processor group=A and in next UpdateAttribute processor set the following:

value1=${ group:prepend('hash{ '):append('_value1 }'):replace('hash', '#'):evaluateELString() }

The magic being done here is "Take value of group slap around it #{ and _value1 } to make it valid NiFi Expression Language statement and then evaluate it." (Notice - the word hash and function replace is there since I didn´t manage to escape the # char right before {.)

If you would like to have your value1 at the beginning of the statement then you can use following code. The result is same, it is easier to use (often-changed value value1 is at the beginning of the statement) and is less readable "what is really going on?"-wise.

value1=${ literal('value1'):prepend('_'):prepend(${ group }):prepend('hash{ '):append(' }'):replace('hash', '#'):evaluateELString() }

Upvotes: 2

Related Questions