CODEWITHSUNDEEP
csvapache-nifi
Sachith Muhandiram
Sachith Muhandiram

Reputation: 2972

Record processing using apache NiFi

I have a csv with following data :

id,name,loan,to_pay,paid
1,sachith,"house,cc,vehicle","10000,24500,233300","2000,5000,2030"
2,nalaka,"edu,cc","3400,5000","200,450"

Here one user can have multiple loans (max 5). I want to split this into multiple rows.

id,name,loan,to_pay,paid
1,sachith,house,10000,2000
1,sachith,cc,24500,5000
1,sachith,vehicle,233300,2030
2,nalaka,edu,3400,200
....

Is there a specific processor for doing this?

Edit : Warning message in ExecuteGroovyScript after @daggett answer:

GroovyScript is invalid because
ord.codehaus.groovy.control.MultiCompilationErrorsException: startup failed:
General error during conversion : Error grabbing Grapes  --[unresolved dependency: org.apache.commons#commons-csv;1.8not found] java.lang/RuntimeExceptopm

Upvotes: 0

Views: 352

Answers (1)

daggett
daggett

Reputation: 28564

use ExecuteGroovyScript processor:

//for complex csv let's use apache library for parsing/printing
@Grab(group='org.apache.commons', module='commons-csv', version='1.8')
import org.apache.commons.csv.*

def ff=session.get()
if(!ff)return

ff.write{streamIn, streamOut->
    streamOut.withWriter("UTF-8"){w->
        def printer = CSVFormat.EXCEL.withHeader("id","name","loan","to_pay","paid").print(w)
        streamIn.withReader("UTF-8"){r->
            def parser = CSVFormat.EXCEL.withHeader().parse(r)
            parser.each{row->
                //split multivalues and print them in a loop
                def loan   = row.loan.split(',')
                def to_pay = row.to_pay.split(',')
                def paid   = row.paid.split(',')
                for(int i=0;i<loan.size();i++){
                    printer.printRecord(row.id, row.name, loan[i], to_pay[i], paid[i])
                }
            }
        }
        printer.flush()
    }
}
REL_SUCCESS<<ff

Upvotes: 1

Related Questions