CODEWITHSUNDEEP
springmultithreadingparallel-processingspring-batch
John
John

Reputation: 121

Multi-threading and Parallel Processing in Webapplication - Advise needed

Question: In a webapp, we have to call a Webservice with Input as N(100) number so that the webservice would get the responses from the queue. Once I get the 100 responses I have to divide it in 5 threads and divide this responses accordingly (20 each) for each thread. And once the processing of the thread is done, it should again call the webservice.

I have the webservice which would return responses depending on the Input number but I don't know what technology I should use to implement the other part.

I have researched on following:

  1. Spring Task Execution: http://static.springsource.org/spring/docs/3.0.x/reference/scheduling.html - but this won't fit as I can't specify how many threads I need to start.

  2. Partitioning: section7.4: http://static.springsource.org/spring-batch/reference/html/scalability.html - I just came across this but haven't worked with this earlier.

It would be great if any one of you could guide me in the right direction.

This is my first post so I apologize if there are any mistakes.

Update: 11/21/2012: I have implemented with the ThreadPoolTaskExecutor. I will post my code snippet after the vacation. But one thing that bugs me is I have to divide the 100 results into group of 20 and have the taskExecutor.submit(); in the For Loop to loop for 5 times which would create 5 threads. It would be nice to have a straight forward way where I submit this 100 requests and can choose how many threads I would like to spawn.

Happy ThanksGiving guys. will update the post after holidays

Upvotes: 2

Views: 2895

Answers (3)

Isaac
Isaac

Reputation: 16736

If you don't already use Spring, then I wouldn't use Spring just for its task execution framework: I would use standard Java for this. Take a look at executors.

Upvotes: 1

Serkan Arıkuşu
Serkan Arıkuşu

Reputation: 5609

For using Spring Task Execution, you mentioned "but this won't fit as I can't specify how many threads I need to start."

Actually using ThreadPoolTaskExecutor you can achieve this quite easily.

<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
  <property name="corePoolSize" value="5" />
  <property name="maxPoolSize" value="10" />
  <property name="queueCapacity" value="25" />
</bean>

<bean id="taskExecutorExample" class="TaskExecutorExample">
  <constructor-arg ref="taskExecutor" />
</bean>

The corePoolSize attribute here sets the number of threads to start initially.

You may read 25.2.2 Using a TaskExecutor in http://static.springsource.org/spring/docs/3.0.x/reference/scheduling.html

Upvotes: 0

r0ast3d
r0ast3d

Reputation: 2637

You can try to see if google guava would fit http://code.google.com/p/guava-libraries/wiki/ListenableFutureExplained or if the http://akka.io/ framework would fit for your needs. Both have ease of encapsulating thread handling and call back implementation.

Upvotes: 1

Related Questions