How to implement distributed end-to-end transactions with JMS

Question

I need to implement a transaction that spans applications running on different systems and communicating via JMS. Actually, I don't need to implement it yet, at this point I just need a high level design with some confidence that it can be implemented.

Here's what I need to do.

Assume I have two applications (running on different servers) that are maintaining some state information that must be kept consistent. For simplicity, let's say they are both keeping track of the value of a single variable X.

I need the sending application to do the following:

• Send a message to peer requesting that X be set to some new value.
• If the peer accepts the change, then make the change locally.
• If the peer rejects the change, then leave the value of X as it is.
• If a communications error or timeout occurs, then leave X as it is.

The receiver has to do something similar, and the whole thing has to be "transactional" in the sense that I need high confidence that both systems have the same value of X, changed or not, within some fixed time interval. I can afford rare cases where things break down, as long as the software on at least one end knows that things have run amok and can alert the operator that the value of X may be inconsistent.

At first I thought this would be relatively easy using the features of JMS. But after doing a bunch of reading about JMS, now I don't think so. There are some complications, in particular:

• This is a near-real time application. If the change cant be made on both sides within a fixed (short) time, then I need to abort the change.
• I don't have a single JMS queue (or topic) between the sender and receiver. Instead, I have other processes (Oracle Service Bus stuff) that are moving messages from the JMS queue used by the sending application to a JMS queue read from by the receiving application. The intermediate process may be doing things like message format transformation, and it might also be mediating between different JMS implementations (sender is using one vendor's JMS, receiver is using another's). This means that there is a chance things can go wrong in the middle - messages can get delayed, lost, delivered out of order, and so on. I can't rely on JMS delivery guarantees end-to-end.

I read up on how JMS acknowledgements work, the different acknowledgement modes, and using JMS transactions. Tell me if I'm wrong, but I don't think any of this is going solve my problem. These features may allow me to handle any errors that would occur in getting my message gets delivered from my app to the JMS broker, but:

• won't give me anything end-to-end
• won't do anything about problems that occur inside the broker (e.g. queue overflows, message TTL expiration, etc.)

So then I started reading up on Distributed Transaction Processing and XA. In theory, I think this is what I need. A single transaction manager and a 2-phase commit protocol. However, this is not something that is feasible to implement myself, I would need to use an off-the-shelf solution. So I'm reading up on JTA, but I'm still struggling to understand the concepts. It seems that JTA allows me, from one application, to have a single transaction that involves a bunch of different operations, e.g. send a JMS message, write data to a database, etc... If each of these things has an XA compliant "resource manager" then I can put them all together into a single transaction (everthing will happen, or it will all get rolled back) from my one application using JTA.

But I'm struggling to see how I can use that. I don't have one application, I have two. And I don't have an XA compliant resource manager for these applications. Would I have to write one? Somehow use an application that does (e.g. a database)?

Any suggestions?

Thanks and sorry for the newbie questions!

Duncan

Answer 1

Transactions, even distributed transactions, often just makes sure some messages get written to queue, read from a queue or a message processing commits at the same time as some database operation. It won't solve your problem unless you add a lot of complexity.

Your problem looks like a data synchronization problem over multiple systems. I recommend looking at some other way to handle that synchronization.

One common approach is to write the latest update of some value (price?) to a JMS topic together with some timestamp (or other information to consider if this information is valid). Then all systems can keep up with the latest information (and safely reject publications with "out-of-date" information). Can something similar solve your case?

If your different systems have different underlying databases, you can of course use XA transactions (there are open source XA transaction managers around) and commit the new value X to both databases. This will be "safe", but on the other hand, it will keep your system tightly coupled, which often is not a good idea.