CODEWITHSUNDEEP
hadoopooziesqoop
Sedat Kestepe
Sedat Kestepe

Reputation: 95

sqoop oozie write query result to a file

I have a current oozie job that queries an Oracle table and writes - overwrites the result on a hive query. Now I need to prevent overwriting the hive table and save the existing data on that hive table. For this I wanted to plan such steps: 1st step: Get record count running a "select count(*) from..." query and write it on a file. 2nd step: Check the count written in file. 3rd step: decision step whether or not 4th step will be applied. 4th step: Run the main query and overwrite the hive table.

My problem is I couldn't find anything on documentation and or examples regarding writing them on a file (I know import and export is the aim of sqoop) . Does anyone know how to write the wuery result on a file?

Upvotes: 0

Views: 551

Answers (1)

Samson Scharfrichter
Samson Scharfrichter

Reputation: 9067

In theory:

  1. build a Pig job to run the "count(*)" and dump the result to StdOut as if it was a Java property e.g. my.count=12345
  2. in Oozie, define a Pig Action, with <capture_output/> flag, to run that job
  3. then define a Decision based on the value for key my.count using the appropriate EL function

In practise, well, have fun!

Upvotes: 1

Related Questions