CODEWITHSUNDEEP
csvapache-camelflat-file
Marcin Wiśniewski
Marcin Wiśniewski

Reputation: 253

Apache camel + csv + header

I have csv file as follows:

A;B;C
1;test;22
2;test2;33

where first line is a kind of header, and others are data. I have an issue to import all data rows with respect to header and report how many rows are correct and how many are not.

My first idea is to split source file to multiple files in the form of:

file1:

A;B;C
1;test;22

file2:

A;B;C
2;test2;33

How can I do this in camel, and how can I collect data necessary to print a summary report?

Upvotes: 1

Views: 4758

Answers (3)

brunocrt
brunocrt

Reputation: 780

You can unmarshall the file as a CSV file, remove the first line (header) and then do your validations as desired. Follow an example of camel route implementation

from("file:mydir/filename?noop=true")
.unmarshal()
.csv()
.process(validateFile())
.to("log:my.package?multiline=true")

Then you need to define the validateFile() method using the camel Processor class like this:

public Processor validateFile() {
    return new Processor() {
        @override
        public void process(Exchange exchange) throws Exception {
            List<List<String>> data = (List<List<String>>) exchange.getIn().getBody();
            String headerLine = data.remove(0);
            System.out.println("header: "+headerLine);
            System.out.println("total lines: "+data.size());

            // iterate over each line
            for( List<String> line : data) {
                System.out.println("Total columns: "+line.size());
                System.out.println(line.get(0)); // first column
            }
      }
    };
 }

In this method you can validate each file line/columns as you wish and then print it out or even write this report in other output file

Use as reference the File and CSV component page from Apache camel docs;

Upvotes: 0

Daniel
Daniel

Reputation: 2500

Take a look at Bean IO, and the Camel BeanIO component. Looks like a good fit for your scenario. You could probably build upon the example code on the first page of bean IO

BeanIO http://beanio.org/

Camel BeanIO component http://camel.apache.org/beanio.html

You should not need to split your incoming file if the only thing you need to do is collect and count successful and unsuccessful records.

Upvotes: 3

G Quintana
G Quintana

Reputation: 4667

If the CSV is not too big and fits in memory, I would read and convert the CSV file to a list of Java objects. The latest Camel CSV component can convert a CSV file into a List<Map>, before Camel 2.13 it produced List<List>. After having read converted CSV file into List of something you can write your own processor to iterate over the List and check its content.

Upvotes: 0

Related Questions