CODEWITHSUNDEEP
javafilesplitapache-camel
GLMills
GLMills

Reputation: 638

Apache Camel Message Exchange - can you iterate over the exchange to produce multiple output files from a single message on in exchange

I have a single inbound message on the exchange. This message is split out in to multiple outbound files. The problem I see is that only the LAST file is produced. Logs show that we created successfully all the different files, however, can only write a single file to the exchange, the LAST file processed? please tell me what I'm doing wrong Camelers. thanks in advance!

setting the CamelFileName or the Exchange.FILLE_NAME

Logic that splits out the single file in to multiple files by placing the contents of a specific file in an ArrayList. ...

    log.info("Create output file for current asset: " + asset + " with fileName: " + fileName);
    StringBuilder ofsb = new StringBuilder();
    ofsb.append(outFolder).append("_").append(fileName).append("_").append(TimeStamp()).append(".csv");

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    DataOutputStream out = new DataOutputStream(baos);
    for (String row : serviceRecords) {
        out.writeBytes(row + System.lineSeparator());
    }
    out.flush();

      log.info("**** serviceRecords  " + " file name: " + fileName + "****");
      Iterator<String> pitr = serviceRecords.iterator(); //get the existing service records in array list
      while ( pitr.hasNext()) {
          String sRecord = pitr.next(); 
          log.info(pitr.next());
      }
    exchange.getOut().setHeader("CamelFileName", ofsb.toString());  //CamelFileName Exchange.FILE_NAME
    log.info("Asset File Name: " + exchange.getOut().getHeader("CamelFileName"));
    exchange.getOut().setBody(baos.toByteArray());
    baos.close();
    out.close();
    serviceRecords = new ArrayList<String>();
    sb = new StringBuilder();   
    ofsb = new StringBuilder(); 
    }

multiple files generated and written to file system based on a single inbound file on exchange.

this is a sample of the input file: a previous a process collects all asset relationships, a tag is mapped to an asset. all the tags that belong to a specific asset are made in to a collection. The asset collection is put in an ArrayList. It loops through all the assets related to this input file, creating an out file for each unique asset collection.

1548278588,G1.A_96BD1,1027.8129883,Float,GOOD 1548278590,G1.A_96BD1,1046.7722168,Float,GOOD 1548278591,G1.A_96BD1,1037.0013428,Float,GOOD 1548278596,G1.A_96BD1,1046.5593262,Float,GOOD 1548278598,G1.A_96BD1,1034.9998779,Float,GOOD

THE QUESTION IS HOW DO YOU PRODUCE MULTIPLE FILES FROM A SINGLE EXCHANGE ???

I know something like this works but it is not in line with camel infra.
ofsb.append(outFolder).append("").append(fileName).append("").append(TimeStamp()).append(".csv"); File file = new File(dir, ofsb.toString()); FileWriter fileWriter = new FileWriter(file); CRLFPrintWriter out = new CRLFPrintWriter(fileWriter); for (String outputLine : serviceRecords ) { //otemp, oServiceRecords, seviceRecords, osRecords out.println(outputLine); }

    out.flush();
    out.close();
    fileWriter.close();

    serviceRecords = new ArrayList<String>(); //reset serviceRecords record list of output records
    sb = new StringBuilder(); //reset record builder    
    ofsb = new StringBuilder(); //reset file name
    }

---- SNIPIT of Camel XML

<route autoStartup="true" id="core.fleet.asset.splitterRoute">
    <from id="_from4" uri="{{fileEnrichmentEndpoint}}"/>
    <process id="_process4" ref="assetCollectorProcessor"/>
    <process id="_process5" ref="fleetAssetSplitter"/>
    <to id="_splitOut" uri="{{fileSplitDestination}}"/>
</route>

Upvotes: 0

Views: 773

Answers (1)

burki
burki

Reputation: 7025

For the thing you ask (split a message and write parts to files), there is way too much Java code in this question.

Without reading your question in full detail, here is the conceptional answer to your question. 

<route id="split">
    <from uri="[endpoint where full message arrives]"/>
    ...
    <split ...>
        ...
    </split>
    <to uri="direct:writeFiles"/>
</route>

<route id="writeFiles">
    <from uri="direct:writeFiles"/>
    <setHeader headerName="CamelFileName">
        <simple>...</simple>
    </setHeader>
    <to uri="file:directory?option=value"/>
</route>

The route "split" receives the message to split. You can do whatever you need (like your enrichment). At a certain point you use a Splitter to split the message into parts.

After splitting you send the individual message parts (result of the splitter) to another route. In my example it is a direct route, but you can also use a message queue (ActiveMQ) or an in-memory queue (Seda).

The second route ("writeFiles") just receives the message parts and saves them to files. The file component of Camel simply writes the message body to a file.

Upvotes: 1

Related Questions