CODEWITHSUNDEEP
pythonmultithreadingweb
breez
breez

Reputation: 75

python web thread

So I have a simple python cgi script. The web front end is used to add stuff to a database, and I have update() function that does some cleanup.

I want to run the update() function every time something is added to site, but it needs to be in the background. That is, the webpage should finish loading without waiting for the update() function to finish.

Now I use:

-add stuff to db
Thread(target=update).start()
-redirect to index page

The problem seems to be that python does not want to finish the request (redirect) until the update() thread is done.

Any ideas?

Upvotes: 0

Views: 345

Answers (2)

jfs
jfs

Reputation: 414079

  • you could add a database trigger to update db in response to an event e.g., if a specific column has changed
  • start a subprocess e.g., subprocess.Popen([sys.executable, '-c', "from m import update; update()"]). It might not work depending on your cgi environment
  • or just touch update file to be picked up by an inotify script to run necessary updates in a separate process
  • switch to a different execution environment, e.g., some multithreaded wsgi-server
  • as a heave-weight option you could use celery if it is easy to deploy in your environment

Upvotes: 0

S.Lott
S.Lott

Reputation: 391820

That is, the webpage should finish loading without waiting for the update() function to finish

CGI has to wait for the process -- as a whole -- to finish. Threads aren't helpful.

You have three choices.

  1. subprocess. Spawn a separate "no wait" subprocess to do the update. Provide all the information as command-line parameters.

  2. multiprocessing. Have your CGI connect place a work request in a Queue. You'd start a separate listener which handles the update requests from a Queue.

  3. celery. Download Celery and use it to manage the separate worker process that does the background processing.

Upvotes: 1

Related Questions