CODEWITHSUNDEEP
springgroovyapache-camel
Curt
Curt

Reputation: 107

Multiple camel route processing same file

I have 10 routes defined with 2 of them listed here. The other 8 are identical but the from endpoint is a different dir. Each route picks up a file that is located in the dir and is processed by one groovy class. When one file is dropped into one of the dir it works fine, but when a file is dropped into each of the dir's at the same time each thread seems to go haywire. I do receive logging messages that I have picked up each file, but then it seems like each thread uses one process "beginProcess" because its like all my variables defined in groovy are being changed as each file gets picked up. Im not sure if this makes any sence, but can someone tell me that what I am doing below is legal? Basically can multiple routes call one class and if so whether I am doing that correct? If it is then at least I would know that it has to be in my groovy class(but seeing that it works with one file suggest that its not here, but not assuming that at this point). Thanks much!

<camel:camelContext id="myId"
    xmlns="http://camel.apache.org/schema/spring">

    <route>
        <from
            uri="file://directoryStructure1/?move=archive&amp;sortBy=ignoreCase:file:name&amp;readLock=markerFile&amp;readLockCheckInterval=5000&amp;readLockTimeout=10m&amp;maxMessagesPerPoll=1" />
        <process ref="beginProcess"></process>
    </route>

    <route>
        <from
            uri="file://directoryStructure2/?move=archive&amp;sortBy=ignoreCase:file:name&amp;readLock=markerFile&amp;readLockCheckInterval=5000&amp;readLockTimeout=10m&amp;maxMessagesPerPoll=1" />
        <process ref="beginProcess"></process>
    </route>
</camel:camelContext>

    <bean id="beginProcess" class="package.groovy.class">   </bean>

Upvotes: 4

Views: 3938

Answers (1)

Petter Nordlander
Petter Nordlander

Reputation: 22279

Seems like you are using a singleton (single instance) object of your class since that would be default in spring when you define "beginProcess".

That is fine, if the implementaiton of "package.groovy.class" is thread-safe, which seems to be the case, it is not.

Basically, there are a few ways to fix this.

1) Never process concurrent. It is easy to achieve with the SEDA transport that can stack up files on a queue while another file is processed.

<route>
    <from
        uri="file://directoryStructure1/?move=archive&amp;sortBy=ignoreCase:file:name&amp;readLock=markerFile&amp;readLockCheckInterval=5000&amp;readLockTimeout=10m&amp;maxMessagesPerPoll=1" />
    <to uri="seda:process"/>
</route>
<route>
    <from
        uri="file://directoryStructure2/?move=archive&amp;sortBy=ignoreCase:file:name&amp;readLock=markerFile&amp;readLockCheckInterval=5000&amp;readLockTimeout=10m&amp;maxMessagesPerPoll=1" />
    <to uri="seda:process"/>
</route>

<route>
  <from uri="seda:process"/>
  <process ref="beginProcess"/>
</route>

2) Create a new bean every time the beginProcess is called:

 <bean id="beginProcess" class="package.groovy.class" scope="prototype"/>

A good page about spring bean scoping is found here.

3) Make your package.groovy.class thread safe. That is, no fields and variables should be kept as a state or, the state should be possible to share between concurrent requests and be synchronized.

Upvotes: 5

Related Questions