CODEWITHSUNDEEP
fileesbmulecorrelationaggregator
Gabriele Rossi
Gabriele Rossi

Reputation: 11

How to aggregate files in Mule ESB CE

I need to aggregate a number of csv inbound files in-memory, if necessary resequencing them, on Mule ESB CE 3.2.1.

How could I implement this kind of logics?

I tried with message-chunking-aggregator-router, but it fails on startup because xsd schema does not admit such a configuration:

<message-chunking-aggregator-router timeout="20000" failOnTimeout="false" >
            <expression-message-info-mapping correlationIdExpression="#[header:correlation]"/>
</message-chunking-aggregator-router>

I've also tried to attach mine correlation ids to inbound messages, then process them by a custom-aggregator, but I've found that Mule internally uses a key made up of:

Serializable key=event.getId()+event.getMessage().getCorrelationSequence();//EventGroup:264

The internal id is everytime different (also if correlation sequence is correct): this way, Mule does not use only correlation sequence as I expected and same message is processed many times.

Finally, I can re-write a custom aggregator, but I would like to use a more consolidated technique.

Thanks in advance,

Gabriele

UPDATE

I've tried with message-chunk-aggregator but it doesn't fit my requisite, as it admits duplicates.

I try to detail the scenario I need to cover:

  1. Mule polls (on a SFTP location)
  2. file 1 "FIXEDPREFIX_1_of_2.zip" is detected and kept in memory somewhere (as an open SFTPStream, it's ok). Some correlation info are mantained for grouping: group, sequence, group size.
  3. file 1 "FIXEDPREFIX_1_of_2.zip" is detected again, but cannot be inserted because would be duplicated
  4. file 2 "FIXEDPREFIX_2_of_2.zip" is detected, and correctly added
  5. stated that group size has been reached, Mule routes MessageCollection with the correct set of messages

About point 2., I'm lucky enough to get info from filename and put them into MuleMessage::correlation* properties, so that subsequent components could use them.

I did, but duplicates are processed the same.

Thanks again

Gabriele

Upvotes: 0

Views: 2073

Answers (1)

David Dossot
David Dossot

Reputation: 33413

Here is the right router to use with Mule 3: http://www.mulesoft.org/documentation/display/MULE3USER/Routing+Message+Processors#RoutingMessageProcessors-MessageChunkAggregator

Upvotes: 1

Related Questions