CODEWITHSUNDEEP
pythonconcurrencytransactionsredis
chenzhongpu
chenzhongpu

Reputation: 6871

Redis: How to ensure both concurrency and atomic for shopping case?

Suppose that food_dict is dictionary to store items in shopping cart, 

{<food_id>:<count>}

(One cart may contain multiple foods)

I have stored food in redis in advance:

r.hset('food:<food_id>', {'price': <price>, 'stock': <stock>})

When making order of cart, I have to ensure that stock is larger than count.

The basic implement:

for k,v in food_dict.iteritems():
    _stock = int(redis_db.hget('food:' + str(k), 'stock'))
    if v > _stock:
       # I have to rollback the decrement of stock
       break
    else:
       redis_db.hset('food:' + str(k), 'stock', _stock - v)

To rollback, pipeline is needed.

    pipe = redis_db.pipeline()
    for k, v in food_dict.iteritems():
        _stock = int(redis_db.hget('food:' + str(k), 'stock'))
        if v > _stock:
            return
        else:
            pipe.hset('food:' + str(k), 'stock', _stock - v)
    pipe.execute()

For single client, the code above can well-done. When it comes to concurrency:

with redis_db.pipleline() as pipe:
    while 1:
        try:
            pipe.watch(['food:' + str(k) for k in food_dict])
            stock_dict = {}
            for k in food_dict:
                _stock = pipe.hget('food:' + str(k), 'stock')
                stock_dict[k] = _stock
            pipe.multi()
            for k, v in food_dict.iteritems():
                if v > stock_dict[k]:
                    break
                else:
                    pipe.hset('food:' + str(k), 'stock', stock_dict[k] - v)
            pipe.execute()
            break
        except WathchError:
            continue
        finally:
            pipe.reset()

Can this code guarantee that order can be make only when stock is larger than count?

Upvotes: 0

Views: 337

Answers (1)

Itamar Haber
Itamar Haber

Reputation: 49942

Yes, pipe.execute() should error if any watched keys are changed.

However. The WATCH/MULTI/EXEC pattern allows you to implement optimistic locking and the longer your transactions are, more likely are they to fail. Furthermore, given that the food stock counters are "hot", I don't think this approach is effective in your use case.

I would, instead, implement the transaction as a Lua script that first checks the stock for every item. Then I can decide, in that script, whether to commit some of the orders in the cart, all of them, none or anything else.

EDIT

how to be more effective for hot shopping with python only?

I guess you can only make minor optimizations using exclusively Python. The core difference is that right now you're managing the "transaction" in the application, so when there's concurrency it is more likely to rollback. Lua allows you to perform the tx's logic on the server, so you're basically locking everything but the tx will succeed (as long as there's enough food in stock once it starts).

As for minor optimizations, right now it looks like food:* is a Hash key that stores other stuff besides stock - if any of that information also changes, it will roll back the running transactions. You can consider using a dedicated stock key for each food item (e.g. food:apples:stock) and watch it instead.

Since you're trying to commit the entire cart, perhaps you should also consider breaking this to multiple transactions - one for each item in the cart. This would mean that your application will be responsible for retrying on hot items and rolling back committed changes if you decide to abort entirely, which is usually acceptable with shopping carts (think of how many times you've seen products left in a cart or near the cashier in the grocery store).

P.S. Lua and Python are quite similar so I really encourage you to pick it up. It's fun, easy and well worth it since you're already using Redis...

Upvotes: 1

Related Questions