Reading message from Websphere MQ is always destructive call

Question

I have observed that when msg is read from the queue(destructive call) later if TX is rolled back(MDB container managed TX) msg are placed back to the queue(Original or back out queue depending the on backcount & threshold settings for queue). Lets say if something happens(Repeated Network issue or some issue) due to which rollback of TX got failed then Msgs are lost.

Is it true??

Thanks

Answer 1

As I noted in the comments to the earlier question, the short answer is "no, WMQ will not lose those messages if the COMMIT fails" but let us look in a bit more detail.

When the client makes a connection to MQ over the network, what happens under the covers is that MQ starts a process that creates a shared-memory connection to MQ and a socket connection to the client application. The thing holding the transaction open is the shared memory process which is called a 'channel agent' in WMQ terminology.

There are two cases where the COMMIT appears from the application's perspective to have failed. In the first of these, the COMMIT is delivered to the channel agent and executed, but the acknowledgement never makes it back to the application due to a network problem. From MQ's perspective, the message is not 'lost' because the app intentionally committed the GET calls.

The second case is that the network failure occurs before the COMMIT arrives at the channel agent. In this case, the COMMIT truly has failed because it is not possible for the app to recover the connection handle in which the transaction lives. When the channel agent realizes that the TCP connection has failed, it backs out the transaction and closes the channel. The wrinkle in this process is that it may take a while for the channel agent to realize that the connection has died. In fact, the default TCP timeout on most platforms is two hours. Depending on your server's TCP settings, the transacted GET may hold that message under syncpoint for up to two hours, making it appear to have been consumed. This is one reason that using a current level of QMgr and client is highly recommended - they rely less on TCP and more on an internal channel heartbeat protocol to detect lost connections.

From the application's perspective both of these cases appear to be the same - app calls COMMIT and gets back 2009 MQRC_CONNECTION_BROKEN. But in the first case, the message is gone due to the COMMIT and in the second case, the message may be delivered again. Because of this ambiguity, the app must be able to detect and gracefully handle dupe messages.

Now before declaring that this is a defect in MQ, let me point out that this ambiguity of outcomes is inherent when using messaging over the network and not anything specific to WebSphere MQ. In fact, the JMS 1.1 specification addresses this directly in 4.4.13 where it says:

If a failure occurs between the time a client commits its work on a Session and the commit method returns, the client cannot determine if the transaction was committed or rolled back. The same ambiguity exists when a failure occurs between the non-transactional send of a PERSISTENT message and the return from the sending method.

It is up to a JMS application to deal with this ambiguity. In some cases, this may cause a client to produce functionally duplicate messages.

A message that is redelivered due to session recovery is not considered a duplicate message.

So the use of syncpoint can eliminate the case of lost messages because any GET must be followed by a COMMIT in order to permanently remove the message from the queue. By definition, the COMMIT cannot arrive until the application has seen the message. However, syncpoint cannot eliminate the possibility of duplicate messages because if a COMMIT of a GET fails, the app may see the message again. Similarly, if the COMMIT of a PUT fails, the app has no way to know whether the COMMIT failed before or after reaching the channel agent and has no choice but to reconnect and PUT the message again.

If in fact you are experiencing 'lost' messages, the possibilities are that:

1. The putting app failed on the COMMIT but did not put another copy of the message.
2. The GETting app failed on the COMMIT and the message is not lost but actually under syncpoint. In that case, stop any orphaned channels to roll back the transaction.

Other than those two cases, WMQ will not lose that persistent message.

Answer 2

No. Messages are not lost. In a transaction, the messages are removed from the queue only after successful commit. Messages reappear if commit or rollback fails. WebSphere MQ automatically does a rollback if a connection between application and queue manager breaks before a commit or rollback is called. This ensures that messages are not lost.