CODEWITHSUNDEEP
pythondeadlockgeventoursql
jiakai
jiakai

Reputation: 501

how to solve this deadlock with gevent and oursql

I have the following code from a complex project:

from gevent import monkey
monkey.patch_all()
import gevent

import oursql

def run(num):
    conn = oursql.connect(host = ...)
    cursor = conn.cursor()
    cursor.execute('start transaction')
    for i in range(2):
        print num, i
        cursor.execute('UPDATE userobj SET timestamp=(timestamp + 1) WHERE id = 1')
        gevent.sleep()

    cursor.execute('rollback')


proc = [gevent.spawn(run, i) for i in range(2)]
gevent.wait(proc)

The engine is InnoDB and the output is:

0 0
1 0

Then the program hangs. I know the reason is mysql locks the row after the first greenlet executes the update statement, so the update in the other greenlet will block. But why does gevent not transfer control back to the first greenlet after the other one blocks on socket? And I wonder is there any graceful solution besides using a lock or committing before gevent.sleep?

p.s. The original situation is in a website project. I mixed pymongo and SQLAlchemy operations and used gunicorn to serve the site. But I found that parallel requests may block forever. After a long time of debug, I finally figured out this was due to pymongo used some socket operations, which caused gevent to switch to another greenlet, resulting a deadlock as the above code demonstrates.

Thanks!

Upvotes: 1

Views: 1052

Answers (1)

jiakai
jiakai

Reputation: 501

Thanks to @robertklep , one possible solution is to replace oursql with some pure-python drivers such as pymysql. In fact oursql is written in C and thus incompatible with gevent.

Upvotes: 1

Related Questions