CODEWITHSUNDEEP
componentsspring-integrationworker-thread
prasadsh
prasadsh

Reputation: 35

Should I use spring-integration or simple Spring IoC for worker threads?

We are in the design phase of our application. We have decided to use Spring-Integration. As a starting point, the application reads messages from the JMS queue using Inbound Adapter which is polling based and multi-threaded using task-executor. After audit logging of the message, these receiver threads drop the messages onto the channels from where the worker thread picks up each message for further processing. The further processing itself includes different components like message parsing, node object building, pre-linking and linking. At steps it includes inteceptors for message saving and the node object saving. This is as per the below configuration.

<int-jms:inbound-channel-adapter jms-template="jmsTemplate"  channel="channel1" id="MessageReceiver">
    <int:poller fixed-delay="100" time-unit="MILLISECONDS" task-executor="taskExecutor"/>
</int-jms:inbound-channel-adapter>
<int:channel id="channel1">
        <int:interceptors>
            <int:wire-tap channel="channel.tbl_message"/>
        </int:interceptors>
    </int:channel>

<bean id="dataSource" destroy-method="close"
    class="org.apache.commons.dbcp.BasicDataSource">
    <property name="driverClassName" value="${db.driverClassName}" />
    <property name="url" value="${db.host.url}" />
    <property name="username" value="${db.username}" />
    <property name="password" value="${db.password}" />
</bean>

<int-jdbc:outbound-channel-adapter channel="channel.tbl_message" data-source="dataSource"
    query="#{message.receiver.insert.query}" id="MessageLogger"></int-jdbc:outbound-channel-adapter>

<int:transformer input-channel="channel1" output-channel="channel2" method="parse" id="NodeBuilder">
    <bean class="com.recon.parser.NodeBuilder"></bean>
</int:transformer>
<int:channel id="channel2"/>

<int:filter input-channel="channel3" output-channel="channel5"
    discard-channel="channel4" method="validate" id="NodeValidator">
    <bean class="com.recon.util.Validator"></bean>
</int:filter>
<int:channel id="channel3">
    <int:interceptors>
        <int:wire-tap channel="channel.tbl_node"/>
    </int:interceptors>
</int:channel>
<int-jdbc:outbound-channel-adapter channel="channel.tbl_node" data-source="dataSource"
    query="#{valid.node.insert.query}" id="ValidNodePersist"></int-jdbc:outbound-channel-adapter>

<int:channel id="channel4"/>
<int-jdbc:outbound-channel-adapter channel="channel4" data-source="dataSource"
    query="#{validation.failure.insert.query}" id="FailedNodePersist"></int-jdbc:outbound-channel-adapter>

<int:transformer input-channel="channel2" output-channel="channel3" method="nodeEnricher" id="NodeEnricher">
    <bean class="com.recon.processor.NodeEnricher"></bean>
</int:transformer>

<int:channel id="channel5"/>
<int:service-activator input-channel="channel5" id="LinkerManager">
    <bean class="com.recon.manager.LinkerManager"></bean>
</int:service-activator>

Now I have two options: 1) Create a thread pool of worker threads. Each worker thread will process the message from node processor onwards which will use all the subsequent components in the simple spring's dependency injection way without using spring-integration. 2) Create a threadpool of each component using task-executor. Each of the threads of each component will pick the input object buffered onto its previous channels. However, this will create many number of threads as each component is going to be multithreaded.

Could anyone please suggest some solution on the approach to be selected?

Upvotes: 0

Views: 377

Answers (2)

Rajesh Gheware
Rajesh Gheware

Reputation: 157

@prasadsh Are you getting required performance from by converting <wire-tap> channels to QueueChannel? You can also try converting the channel1 to be a QueueChannel too as you anyway you have channel.tbl_message to record the input messages from jms.

I am curious to know if you have got any solution working because we have similar requirements (10k messages/sec)

Upvotes: 0

Artem Bilan
Artem Bilan

Reputation: 121177

As soon you start your flow from JMS message poller, I don't see reason to shift messages from the polling Thread to any other one. Would be better, if your general logic will be done within that single thread and in this case the throughput of your application will depend on the Poller TaskExecutor concurrency.

audit logging, message saving and node object saving might be done from different thead. In this case the QueueChannel is for you. I mean all those <int:wire-tap> channels.

Upvotes: 0

Related Questions