CODEWITHSUNDEEP
apacheapache-camel
sandip desale
sandip desale

Reputation: 61

apache camel split csv file

We are having a requirement to split a large CSV into smaller multiple CSV files. First row of main CSV file contains the column header which I need to have in all generated CSV files.

How to achieve it?

I am able to split file using Apache Camel Splitter route as follows... 

        <split id="LineItemSplitter" streaming="true"
            parallelProcessing="true" timeout="0">
            <tokenize token="\n" group="5000" />
            <to id="LineItemOutbox"
                uri="file:/target?fileName=${file:name.noext}-${date:now:yyyyMMddHHmmssSSSSS}.csv?fileExist=Append" />
        </split>

    </route>

Upvotes: 2

Views: 2638

Answers (1)

isaac.hazan
isaac.hazan

Reputation: 3874

One possible way to handle that is to use the aggregator pattern and have the column headers added in the strategy class.

For example see below:

        <aggregate strategyRef="messageAggregatorStrategy">
            <correlationExpression>
                <constant>true</constant>
            </correlationExpression>
            <completionTimeout>
                <simple>10000</simple>
            </completionTimeout>
            <completionSize>
                <simple>500</simple>
            </completionSize>
            <to id="LineItemOutbox"
                  uri="file:/target?fileName=${file:name.noext}-${date:now:yyyyMMddHHmmssSSSSS}.csv?fileExist=Append" />
        </aggregate>

Then your strategy bean:

<bean id="messageAggregatorStrategy" class="com.youorg.MessageAggregator" />

Then in your strategy you can set the column header:

public class MessageAggregator implements AggregationStrategy
{
    private static final Logger logger = Logger.getLogger(MessageAggregator.class);
    private String COLUMN_HEADERS = "a,b,c";

    @Override
    public Exchange aggregate(Exchange oldExchange, Exchange newExchange)
    {
        Exchange exchangeToReturn;
        StringBuffer csvBuffer;
        String csv = newExchange.getIn().getBody(String.class);
        if (oldExchange == null)
        {
            csvBuffer = new StringBuffer();
            csvBuffer.append(COLUMN_HEADERS).append("\r\n");
            csvBuffer.append(csv);
            newExchange.getIn().setBody(csvBuffer);
            exchangeToReturn = newExchange;
        }
        else
        {
            csvBuffer = (StringBuffer) oldExchange.getIn().getBody(StringBuffer.class);

            // update the existing message with the added body
            csvBuffer.append("\r\n");
            csvBuffer.append(csv);
            oldExchange.getIn().setBody(csvBuffer);

            // and return it
            exchangeToReturn = oldExchange;
        }
        return exchangeToReturn;
    }

The drawback of this approach might performance because you re-aggregate instead of direcly writing to the target file but it does achieve what you are looking for.

Hope that helps.

Upvotes: 1

Related Questions