CODEWITHSUNDEEP
pythondjango
Rachan
Rachan

Reputation: 27

Running "tasks" periodically with Django without seperate server

I realize similar questions have been asked however they have all been about a sepficic problem whereas I don't even now how I would go about doing what I need to.

That is: From my Django webapp I need to scrape a website periodically while my webapp runs on a server. The first options that I found were "django-background-tasks" (which doesn't seem to work the way I want it to) and 'celery-beat' which recommends getting another server if i understood correctly.

I figured just running a seperate thread would work but I can't seem to make that work without it interrupting the server and vice-versa and it's not the "correct" way of doing it.

Is there a way to run a task periodically without the need for a seperate server and a request to be made to an app in Django?

Upvotes: 1

Views: 215

Answers (2)

Josh Garrett
Josh Garrett

Reputation: 48

I'm not sure if this is the "correct" way but it was a cheap and easy way for me to do it. I just created custom Django Management Commands and have them run via a scheduler such as CRON or in my case I just utilized Heroku Scheduler for my app.

Upvotes: 1

sytech
sytech

Reputation: 40861

'celery-beat' which recommends getting another server if i understood correctly.

You can host celery (and any other needed components) on the same server as your Django app. They would be separate processes entirely.

It's not an uncommon setup to have a Django app + celery worker(s) + message queue all bundled into the same server deployment. Deploying on separate servers may be ideal, just as it would be ideal to distribute your Django app across many servers, but is by no means necessary.

Upvotes: 1

Related Questions