mr rogers
mr rogers

Reputation: 3260

Is python `threading.local` safe across requests with `Gunicorn` and `eventlet`?

We're running a Django project with gunicorn and eventlet.

I'd like to use threading.local to stash some http request data for use later in that thread (with some custom middleware). I'm wondering if this is safe with eventlet.

From the docs

Eventlet is thread-safe and can be used in conjunction with normal Python threads. The way this works is that coroutines are confined to their ‘parent’ Python thread. It’s like each thread contains its own little world of coroutines that can switch between themselves but not between coroutines in other threads. which sounds like it might be.

But I understand that eventlet, based on reading their docs on 'How the Hubs Work', may suspend a co-routine to process another one. Is it possible, with gunicorn the an http request processing may get suspended and another http request would get picked up and processed by a co-routine in that same initial thread? And if so, does that mean that the threading.local could get shared between two requests?

Can I get away with using threading.local and be certain that each incoming request will get it's own thread.local space?

I also saw this post

the simultaneous connections are handled by green threads. Green threads are not like real threads. In simple terms, green threads are functions (coroutines) that yield whenever the function encounters I/O operation

which makes me think a single "thread" could process multiple requests. And I guess if that is true, then I wonder where exactly is threading.local? at the thread? in a co-routine eventlet (air quotes)thread(air quotes)?

Any pointers would be appreciated here.

Thanks

Upvotes: 2

Views: 729

Answers (1)

mr rogers
mr rogers

Reputation: 3260

tl;dr: the answer is yes.

The eventlet coroutines are treated as separate threads so threading.local will work.

A longer discussion is available on the eventlet GitHub issue.

Upvotes: 2

Related Questions