CODEWITHSUNDEEP
spring-integration
Mark1234
Mark1234

Reputation: 619

configure a task executor for parellel processing of intemediate steps under transaction

The SI workflow starts with an inbound-channel-adapter which runs under a new transaction started by a pollar. The adapter triggers a data processing flow which kind of fans out, for example the adapter polls a database and get few rows, splits and next adapter makes another db call (hence for each row) as input which produces several other rows and so on. Right now its running as single threaded behavior as I only want to commit the original transaction when everything went well. Now I want to to speed up the processing by running it under more threads, so if one of my original adapter produced 3 rows, I want to process them simultaneously for downstream flow and so on. is this possible and if yes, how can I define a global task executor with some configuration and allow the processing at various stages to be executed under this.

Thank you

Upvotes: 0

Views: 337

Answers (1)

Gary Russell
Gary Russell

Reputation: 174544

You could insert a <gateway/> in the flow between the inbound adapter and the rest of the flow; the poller thread will be suspended in the gateway awaiting a reply. You would likely need an aggregator to send the reply to the gateway when the other tasks are complete. You would discard the reply.

<service-activator input-channel="fromAdapter" ref="gw" output-channel="nullChannel" />

<gateway default-request-channel="toRestOfFlow" />

The problem is if the reply is never received the thread will wait forever. If you add a reply-timeout, the transaction will commit after the timeout. So you may need some additional logic to handle that, perhaps adding your own bean that invokes the gateway and detects a null reply.

Of course, in all these scenarios; the async tasks cannot participate in the original transaction.

Upvotes: 1

Related Questions