CODEWITHSUNDEEP
apache-nifi
Neil P
Neil P

Reputation: 3210

NiFi - Call Rest API for every row in the file

I have a datset of IDs, I've got a flow file that has one row per ID. I have an API that takes this ID as a parameter, and I want to harvest the results for all rows back into NiFi (example below).

https://service.com/api/thing/{ID}

How in NiFi, can I call this API, for all IDs in my dataset. Ideally using some parallelism if possible.

(for reference, in SSIS I could load these IDs into an array and then loop over an API call with a parameter for the ID).

Upvotes: 0

Views: 685

Answers (1)

yaprak
yaprak

Reputation: 547

First, use SplitText to get each Id as a flowfile enter image description here

Then copy content to an attribute by ExtractText . add custom property such as 'message.body' in this example

enter image description here

so that ExtractText would add message.body.0 attribute to the flowfile and you can use it InvokeHttp like below . Please note that since your endpoint is https , you may need to configure SSL Contect Service

enter image description here

Finally , you can set concurent task count for each Processor for parallelism

enter image description here

Upvotes: 1

Related Questions