CODEWITHSUNDEEP
hadoophivehiveqlapache-nifi
Jorge Machado
Jorge Machado

Reputation: 772

Run Multiple hive sql on Apache Nifi

I have a Apache Nifi 1.2 workflow that reads from hdfs process some stuff and writes the information on a history table. After that I want to run 3 hive querys to create a new table based on the history table. Why is it so hard to do it on Nifi ? I did not find any simple way to use the putSQL or SelectHiveQL. I don't want to use ExecuteStreamedCommand because I want to keep to the nifi libs.

Any help on this ? thanks

Upvotes: 1

Views: 1400

Answers (1)

RisJi
RisJi

Reputation: 182

Use GenerateFlowFile that contains the first hql you want to run. Connect it to PutHiveQL which is connected to ReplaceText.

In ReplaceText, replace the existing content with 2nd hql and then connect to another PutHiveQL. and so on....

It is very annoying but if you have only 3 hqls then it works.

Update: Better way to do this is to use GenerateFlowFile. Put your entire script there. Connect it to SplitContent and split on semicolon (;) and then send it to PutHiveQL.

Upvotes: 2

Related Questions