CODEWITHSUNDEEP
apache-nifi
John Paulus Prem
John Paulus Prem

Reputation: 31

How to dynamically fetch rows by referencing the table and row id from incoming flow files in nifi

In my use case I will have a hbase table from where I will be reading incrementsal records using getHbase and one of the value from that record should be should be used as a input to FetchHBaseRow processor. So how can i access the file content from the flow file and use it as input to FetchHbaseRow.

Upvotes: 1

Views: 683

Answers (1)

Sivaprasanna Sethuraman
Sivaprasanna Sethuraman

Reputation: 4132

You can have GetHBase and connect it to EvaluateJsonPath (since GetHBase outputs the result in JSON format) to get Row ID and then use the Row ID in FetchHBaseRow by setting the dynamic attribute name used in EvalueteJsonPath in the Row Identifier property in FetchHBaseRow.

In my example, I have mentioned EvaluateJsonPath is used to read the row key, but you can use a valid Json Path expression to read any values from the GetHBase output and use it.

Sample Flow

Sample Flow

Reading Row Key

Using EvaluateJsonPath, we are reading the row key from the JSON output by the GetHBase processor. Ensure that Destination is set to flowfile-attribute

Reading Row Key

Using the parsed Row key

We read the row key and put it in the row.key attribute. Here, we are using that attribute in the property Row Identifier in FetchHBaseRow

Using parsed Row key

Upvotes: 5

Related Questions