CODEWITHSUNDEEP
javaspringapache-camelspring-jms
KChakraborty
KChakraborty

Reputation: 31

Message consumption stuck from activemq using camel jms component

I have an application with two camel routes:

Route 1 (Consumer Route)

Reads a text file having no. of records (line separated), split them based on each line and sends each split record to another queue ('intermediate' queue)

Route 2 (Producer Route)

Reads each record from intermediate queue, transform them and sends them to an out queue.

I am using ActiveMQ with camel JmsComponent (concurrentConsumers=7, maxConcurrentConsumers=10). I guess camel uses Spring DMLC underneath to read from the queue.

Configurations:

activemq.broker.uri = tcp://0.0.0.0:61616?jms.useAsyncSend=true&jms.prefetchPolicy.queuePrefetch=1

<bean id="cachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
    <property name="targetConnectionFactory" ref="jmsmqConnectionFactory" />
    <property name="reconnectOnException" value="true" />
    <property name="sessionCacheSize" value="7" />
</bean>

<bean id="jmsComponent" class="org.apache.camel.component.jms.JmsComponent">
    <property name="connectionFactory" ref="cachingConnectionFactory" />
    <property name="cacheLevelName" value="CACHE_CONSUMER" />
    <property name="transacted" value="true" />
    <property name="transactionManager" ref="transactionManager" />
</bean>

jmsComponent:queue:INTERMEDIATE?concurrentConsumers=7&maxConcurrentConsumers=10

Now, the problem is, if the no. of records in the file is very less (10 or less), the split records are getting stuck in the intermediate queue. The Producer Route is running but no message consumption. There is no exception found in any of the logs and the Consumer Route is also up and running.

However, by setting prefetch limit to 0, this problem is gone but leading to another issue - The camel routes could not be forcibly stopped by ctrl+C with cacheLevel as CACHE_CONSUMER. Though CACHE_AUTO works fine, performance degrades.

Now is there any known issue with prefetch > 0 and SpringDMLC or am I missing something?

Upvotes: 3

Views: 2462

Answers (1)

Paweł Motyl
Paweł Motyl

Reputation: 1332

I know the question is old, but maybe someone will benefit from a workaround. We had encountered the same problem and while we didn't get to the bottom of it (limited resources and need of a prompt fix), we at least isolated Camel as the problem source.

The workaround that we implemented was simply to use two distinct JMS Components - one for Consumer and one for Producer. If there are many Consumer/Producer pairs that produce/consume to/from the same queue you can put all Consumers in one component and all Producers in the other. After introducing the separation we never hit this particular problem again.

Upvotes: 1

Related Questions