CODEWITHSUNDEEP
pythonflask
licorna
licorna

Reputation: 5860

Using threading.Timer after a Request to my Flask Python Application

I want to know if using threading.Timer after receiving a request for executing a task 30 minutes later makes sense. Let's see the code:

def late_process():
    if not finish:
        Timer(1800, late_process, ()).start()
        # Work to do ...
        # Write logs, send emails... whatever

@app.route('/timely-req')
def timely():
    finish = False
    Timer(1800, late_process, ()).start()
    return 'To be executed in 30 minutes'

@app.route('/end-timely-req')
def end():
    finish = True
    return 'Process stopped'

So the main idea is to trigger the execution (every 30 minutes) of this process. This is working but I don't know if threading.Timer is a good idea, as the request will return but I'm leaving the server with a resident thread waking up every 30 minutes. This is only or a prototype to be used and tried and it is not a final solution that will see production some day.

I'm using Python 2.6, Flask 0.10.1 and Uwsgi.

Upvotes: 0

Views: 3108

Answers (1)

abarnert
abarnert

Reputation: 365587

threading.Timer has a bit of a clunky interface (e.g., it doesn't auto-repeat every 30 minutes, you have to fire it off again yourself—and the obvious way to do that means it gradually drifts behind over time).

It's also a bit of a heavyweight implementation; if you have lots of timers, you have lots of threads, which can be a problem for your OS's thread scheduler.

If you've only got a single timer, and you're only running every 30 minutes, and it's all for "background" stuff like log rollover and summary emails, those problems are both probably acceptable—in fact, barely even problems. But there are problems you do have.

First, doing finish = True inside a function creates a local variable named finish, overshading any global variable of the same name. If you want to modify the global variable, you need a global finish statement at the top of the function.*

Also, every time someone hits /timely-req, you're going to fire off a new Timer. Which means I could easily DoS your server—maybe even by accident—just by requesting that URL a few hundred times. So, if you're going to do this, you probably want to make it a singleton object, and create the timer iff it doesn't already exist. There are also dozens of implementations of single-thread multi-timer schedulers on PyPI, ActiveState, and Flask contributions, which would automatically take care of this problem (as well as the two problems above that you probably don't care about).

Also, when you tell Flask to quit, it stops taking requests, lets your code finish off existing requests, lets any threads that you created finish, then quits. If you've got a thread that never finishes, this isn't going to work, so if you want graceful shutdown functionality, you have to add it yourself (setting finish = True).

Finally, it isn't thread-safe to share a global variable between threads without a lock or other thread synchronization device. In particular, it's theoretically possible that your server thread will set finish = True, and your timer thread will keep seeing the old value in the cache forever. In practice, at least with CPython, you will get away with this.** But it's better to write code correctly.

* Also, Flask has an optional feature that fakes globals with thread-local objects—that is, each thread would get its own copy of finish. And, since each Timer instance is a new thread, setting finish = True won't affect it. So, if you're using this feature, you can't use Timer this way. But you probably aren't using it, if you don't know about it.

** In CPython, rebinding a global variable is an atomic operation, but not interlocked—but because of the GIL, it will in practice always be available to every other thread by its next GIL timeslice—which is a fraction of a second, so your 30-minute delay won't even notice. Worst case scenario, it will run one more time than you'd like—which could already happen anyway if the click on the web page took a few milliseconds longer than you expected.

Upvotes: 6

Related Questions