CODEWITHSUNDEEP
apache-kafkaspring-integrationspring-cloud-streamspring-cloud-dataflow
krajwade
krajwade

Reputation: 133

Stream CSV file using Spring Cloud Dataflow with Kafka

I am trying to stream csv file with Spring Cloud dataflow and Kafka. The idea is to stream the file line by line with header included for each line. For example, if the content of my csv file is 

street,city,zip,state
3526 HIGH ST,SACRAMENTO,95838,CA
51 OMAHA CT,SACRAMENTO,95823,CA
2796 BRANCH ST,SACRAMENTO,95815,CA

Then the streamed data should be in following format ( Desired Output ) 

street,city,zip,state
3526 HIGH ST,SACRAMENTO,95838,CA

street,city,zip,state
51 OMAHA CT,SACRAMENTO,95823,CA

street,city,zip,state
2796 BRANCH ST,SACRAMENTO,95815,CA

I am using FileSplitter to pass the messages to the output channel. 

private Source channels;

FileSplitter splitter = new FileSplitter(true, true);
splitter.setOutputChannel(channels.output());
splitter.handleMessage(new GenericMessage<File>(file));

The output that I am getting currently is 

Sink :    street,city,zip,state
Sink :    3526 HIGH ST,SACRAMENTO,95838,CA
Sink :    51 OMAHA CT,SACRAMENTO,95823,CA
Sink :    2796 BRANCH ST,SACRAMENTO,95815,CA

Upvotes: 1

Views: 907

Answers (1)

Gary Russell
Gary Russell

Reputation: 174574

Add a filter after the splitter to drop the first line.

Add a transformer after the filter to transform the payload to

"street,city,zip,state\n" + payload

Upvotes: 1

Related Questions