CODEWITHSUNDEEP
pythondjangolong-running-processes
excieve
excieve

Reputation: 596

How to start a long-running process from a Django view?

I need to run a process that might take hours to complete from a Django view. I don't need to know the state or communicate with it but I need that view to redirect away right after starting the process.

I've tried using subprocess.Popen, using it within a new threading.Thread, multiprocessing.Process. However, the parent process keeps hanging until child terminates. The only way that almost gets it done is using a fork. Obviously that isn't good as it leaves a zombie process behind until parent terminates.

That's what I'm trying to do when using fork:

if os.fork() == 0:
    subprocess.Popen(["/usr/bin/python", script_path, "-v"])
else:
    return HttpResponseRedirect(reverse('view_to_redirect'))

So, is there a way to run a completely independent process from a Django view with minimal casualties? Or am I doing something wrong?

Upvotes: 16

Views: 8575

Answers (4)

Paul McMillan
Paul McMillan

Reputation: 20107

Take a look at the code in kronos.py to see one solution to this problem.

http://www.razorvine.net/download/kronos.py

Upvotes: 2

Oduvan
Oduvan

Reputation: 2757

First of all - try to using cron for you task, as early say shanyu.

If it doesn't suit you - then try to use CeleryProject, for task Queue for Django. For working it uses RabbitMQ. And here is a little overview for simple using of basing futures

Upvotes: 5

Mike
Mike

Reputation: 204

http://code.google.com/p/django-command-extensions/wiki/JobsScheduling

Is a nice library that that you can use to accomplish this task.

Upvotes: 2

shanyu
shanyu

Reputation: 9716

I don't know if this will be suitable for your case, nevertheless here is what I do: I use a task queue (via a django model); when the view is called, it enters a new record in the tasks and redirects happily. Tasks in turn are executed by cron on a regular basis independently from django.

Edit: cron calls the relevant (and custom) django command to execute the task.

Upvotes: 10

Related Questions