CODEWITHSUNDEEP
pythonmultithreadingsqlalchemy
Sindre Smistad
Sindre Smistad

Reputation: 139

Doing threading with sqlalchemy properly?

I have a multi threaded application where the treads work on some objects fetched using sqlalchemy. The objects are put in a thread queue which the threads poll from.

In the main thread I am doing this:

feeds = db_session.query(Feed).filter(Feed.last_checked <= int(update_time)).all()
for feed in feeds:
    self.feed_q.put(feed)

And in the threads I do some updates to the feed objects, and I keep getting these exceptions sometimes when doing updates:

ProgrammingError: (ProgrammingError) (2014, "Commands out of sync; you can't run this command now") 
StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back)

I understand that this has something todo with the threads sharing the same DB session, but I don't know how to fix this.

Upvotes: 1

Views: 2040

Answers (1)

icktoofay
icktoofay

Reputation: 129011

Each thread should have a separate database session. You're probably creating the object that's eventually stored in db_session somewhere, perhaps like this:

db_session = Session()

Essentially, you need each thread to have its own db_session.

Upvotes: 1

Related Questions