How to export data into several Flat Files using Informatica Developer (Big Data Management)?

Question

I have data from Hive table and I want to export the data into several flat files, partitioned by rows, into equal/almost equal number of rows on each target.

Is there any way to partition data into several flat files dynamically like:

• output_01
• output_02
• output_03 and so on, with equal or almost equal number of rows on each target file using Informatica Developer (Big Data Management)?

Answer 1

I can describe how it should be done in PowerCenter or in Informatica Cloud, if you're using any of them.

Short answer is:

• create File Name port in Traget Definition - this will make the dynamic file names possible
• use Transaction Control transformation with TC_COMMIT_BEFORE flag anytime the desired amount of rows gets processed - this will close the current file and redirect writes to new file (with name specified in File Name port value). This will create multiple files as a result.