CODEWITHSUNDEEP
csvmergeapache-nifi
Himsara Gallege
Himsara Gallege

Reputation: 945

Apache Nifi: Merge rows in two csv files

I have two csv files that are funnelled into a MergeContent Processor. I want them to be merged together. They both have the same columns. If the first and second csv's look like this:

First CSV:

id, name
12,John
11,Keels

Second CSV:

id, name
22,Kelly
25,Felder

My output should look like this:

id, name
12,John
11,Keels
22,Kelly
25,Felder

I have tried doing this through the MergeContent Processor. But it Changes the data into a different format I don't want that to happen. Both the Input files and the output files must be .csv and also contain the same name as the input files. (The input files have the same name)

Upvotes: 2

Views: 1536

Answers (1)

Lamanus
Lamanus

Reputation: 13581

Use MergeRecord processor with the common attribute. For example, both flow files have the same attribute such as filename = test.csv then you can set the MergeRecord processor as follows:

Record Reader                      CSVReader
Record Writer                      CSVRecordSetWriter
Merge Strategy                     Bin-Packing Algorithm
Correlation Attribute Name         filename
Attribute Strategy                 Keep Only Common Attributes
Minimum Number of Records          3

The important thing is the minimum number of records, which is the number of rows to be merged. In this case, it should be larger than 2 because each CSV has 2 rows. Then, the CSV will wait for the other CSV to exceed the minimum.

Upvotes: 1

Related Questions