CODEWITHSUNDEEP
scalatransactionsrabbitmqslickatomic
Bob Risky
Bob Risky

Reputation: 935

Can you publish to a queue and write to the db within a Slick transaction, and still guarantee atomicity?

I have a Slick server for which I'd like to use the transactionally keyword to perform a double-write of data to my database and a RabbitMQ message queue. My code looks something like this:

val a = (for {
    _ <- coffees.map(c => (c.name, c.supID, c.price)) += ("Colombian_Decaf", 101, 8.99)
    ch.txSelect()
    ch.basicPublish("SELL " + c.name, QUEUE_NAME, MessageProperties.PERSISTENT_BASIC, "nop".getBytes())
    ch.txCommit()
} yield ()).transactionally

My question is: Is it possible for the queue publish action to commit successfully, but have the DB insert fail? In this case, my system would be in an inconsistent state, as I would only want the message to be published to the queue if the value was successfully inserted into the database, and vice versa.

Thanks!

Upvotes: 1

Views: 1556

Answers (1)

SergGr
SergGr

Reputation: 23798

Unfortunately for you the answer is that you can't easily guarantee consistency for such a system. What you want is distributed transactions and they are fundamentally hard. To see why this is so you can make the following thought experiment: what happens if your computer blows up (or less radically gets cut of electricity) at the most unfortunate moment? For this code one of such bad moments is exactly after the line ch.txCommit() is fully executed (so it is before the outer DB transaction is committed as well). Fundamentally there is nothing you can do about such scenario unless somehow these two concurrent transactions a aware of each other. Unfortunately I don't know about any distributed transaction coordinators to cover both traditional SQL DBs and RabbitMQ. So your choices are:

  1. Give up and do nothing (and develop some procedure to recover after disastrous events afterward in a manual mode)

  2. Implement some distributed transaction algorithm such as 2-phase commit yourself. This most probably requires some re-design and a complicated implementation.

  3. Re-design your system to use some form of eventual consistency. This probably requires a bigger re-design but still might be easier to implement than #2.

Upvotes: 1

Related Questions