CODEWITHSUNDEEP
pythonmultithreadingcherrypybottle
kryptobs2000
kryptobs2000

Reputation: 3537

I can't understand multithreading using cherrypy/bottle

I'm using bottle with a cherrypy server to utilize multithreading. As I understand it this makes each request handled by a different thread. So given the following code:

from bottle import request, route

somedict = {}

@route("/read")
def read():
  return somedict


@route("/write", method="POST")
def write():
  somedict[request.forms.get("key")] = request.forms.get("value")

Would somedict be thread safe? What if a daemon thread were run to manage somedict, say it's a dictionary of active sessions and the daemon thread prunes expired sessions? If not would a simple locking mechinism suffice, and would I need to use it when reading, writing, and in the daemon thread, or just in the daemon thread?

Also as I understand it cherrypy is a true multithreaded server. Is there a more proper method I should use to impliment a daemon thread while using cherrypy as pythons threads are not true threads? I don't wish to delve much into the cherrypy environment preferring to stick with bottle for this project though, so if it involves moving away from bottle/migrating my app to cherrypy then it doesn't really matter for now. I'd still like to know though as I didn't see much in their documentation on threads at all.

Upvotes: 2

Views: 2736

Answers (3)

ron rothman
ron rothman

Reputation: 18148

In your particular example, yes, the (single) dict assignment you perform is threadsafe.

somedict[request.forms.get("key")] = request.forms.get("value")

But, more generally, the proper answer to your question is: you will indeed need to use a locking mechanism. This is true if, for example, you make multiple updates to somedict while handling a single request, and you need them to be made atomically.

The good news is: it's probably as simple as a mutex:

from bottle import request, route
import threading

somedict = {}
somedict_lock = threading.Lock()

@route("/read")
def read():
    with somedict_lock:
        return somedict

@route("/write", method="POST")
def write():
    with somedict_lock:
        somedict[request.forms.get("key1")] = request.forms.get("value1")
        somedict[request.forms.get("key2")] = request.forms.get("value2")

Upvotes: 1

jwalker
jwalker

Reputation: 2009

CherryPy is based on Python threads, so you should stay away from using it as an HTTP server only (and any other native HTTP server). I suggest that you go with uWSGI, which is multiprocess and thus doesn't have GIL issues. Since it is multiprocess, you won't be able to use simple thread-shared variables. You can use uWSGI's SharedArea though or any 3rd party data storage.

Upvotes: 0

korylprince
korylprince

Reputation: 3009

I had originally answered that a dict is threadsafe, but on futher research, that answer was wrong. See here for a good explanation.

For a quick explanation, imagine two threads running this code at once:

d['k'] += 1

They might both read d['k'] at the same time, and thus instead of being incremented by 2, be incremented only by 1.

I don't think it's an issue of your application locking up, more of just some data being lost. If that's not acceptable, using threading.Lock is pretty easy, and doesn't add that much overhead.

Here's some good info on thread safety with CherryPy. You might also consider using something like gunicorn in place of CherryPy. It has a worker process model, so each somedict would be different for every process, so there would be no worry of thread-safety.

Upvotes: 0

Related Questions