CODEWITHSUNDEEP
javajakarta-eejbossjmsmessage-queue
iefpw
iefpw

Reputation: 7042

Batch/bulk processing in JMS

I'm having the following scenario where I receive an XML with multiple entries in the 100,000s. For each item, I have to process the message simultaneously and once that batch/bulk finishes, I have to notify the client that I processed the 100,000 entries that it sent. I'm thinking of adding each message in Message Driven Bean to simultaneously process each one of them. My problem is how do I know that the MDB processed all the messages in this batch, and send me a signal that all the messages in this batch/XML are finished? What is the best way to do this in Java Message Queue? I want to get notified when all the messages in this XML are processed so that I can notify the client.

Upvotes: 0

Views: 1671

Answers (1)

Justin Bertram
Justin Bertram

Reputation: 34988

There are a number of ways this could be implemented but if you want to do it via JMS then I think the simplest and most straight-forward way to do it would be to:

  1. Send a message to a queue for every entry in your XML; call this workQueue.
  2. A consumer (or pool of consumers, it could be an MDB) would take messages from workQueue & process them
  3. Once the consumer was finished processing a message it would place a message on another queue indicating it was finished; call this resultsQueue.
  4. The original sender could listen to resultsQueue and update the client about what entries have (or have not) been processed.

One JMS message per XML entry is pretty granular so you might want to batch multiple XML entries together into a single JMS message to increase throughput if you find that performance doesn't meet your requirements.

Upvotes: 3

Related Questions