Atomicity: Database transaction and Windows Azure

Question

Background

There's a message queue domain. This domain may work with some message queue technology. In this particular case it's bound to Windows Azure Service Bus.

Some project has an operation that registers a Windows Azure Service Bus topic (basically, a message queue).

Registering a new message queue means:

1. Starting a domain transaction (i.e. database transaction).
2. Adding a message queue domain object with some associated info (like what user registers it or to what application belongs the whole message queue).
3. Using inversion of control to handle actual queue registration in message queue server (in this case it'll happen against Windows Azure Service Bus).
4. If nothing went wrong, domain transaction gets comitted and associated domain objects are persisted or updated.

Above background it should work fine if domain transaction ends successfully and Windows Azure Service Bus topic registration worked fine.

Issues

But what happens if domain transaction ends successfully but Windows Azure Service Bus couldn't register the topic - aka message queue -?

Yes, you've a broken domain.

And what happens if you make the Windows Azure Service Bus topic registration before the domain transaction starts? Well, if everything goes fine, no problem. But now the problem has been inverted: what happens if Windows Azure Service Bus topic gets correctly registered but later the domain transaction fails?

Right, this second case is better than first described one, because the operation has registered an useless message queue but domain won't know about it . Mainly the problem now is that an indeterminate number of transactions may fail and an unknown number of message queues would be there just for nothing.

Just imagine if this is a serious and large business service: a lot of useless message queues and a system administrator (a human) deleting useless message queues every month.

So what - now the question -?

Since I find the option of having useless message queues better than having a broken domain, I've opted-in on going forward on this way, but...has Windows Azure any kind of distributed transaction mechanism in order to include any remote object creation in an atomic transaction including other domain-specific operations?, If not - sadly I believe that this is the actual situation -, how would you solve this scenario?

Note:

I'm looking for something like this because I really don't like to have a domain that can't handle its use cases at all. I prefer to avoid this and let domain work properly and have everything in control.

Some interesting link...

I've found some interesting link about this topic:

http://blogs.msdn.com/b/clemensv/archive/2012/07/30/transactions-in-windows-azure-with-service-bus-an-email-discussion.aspx

Answer 1

You can solve this without transactions by using states. Imagine the following workflow:

1. Register the domain and set its state to pending
2. Create the Service Bus Topic
3. Set the state of the domain to active

Then you can have a small process that checks the pending domains:

• If a domain is pending but the topic exists, set the state to active (in this case, step 3 probably failed).
• If a domain is pending but the topic does not exist, try to recreate the topic or send an alert to someone who can do a manual check of what's happening

With a retry policy in place (like TOPAZ), you won't have any issues in 99% of the requests. But whenever something goes wrong, it will get fixed automatically or it will escalate to a human intervention if it can't be fixed.

The alternative would be something like in Clemens' blog post, where you use an outbox table in SQL Azure and you write to this outbox table in the same transaction where you register the domain.