CODEWITHSUNDEEP
pythondjangoscrapy
user2492364
user2492364

Reputation: 6703

How can I seperate the objects in database and run the code by day?

I have too many data to run,so I want to use 20 days to run them

Here is my thought:

place = Place.objects.count()  #place = 101
seperate_to_n_day = 20
print place/20  # 5

and then I have to run this:

place_list = Place.objects.filter(id__gt=0,id__lte=5)       #this month day1
place_list = Place.objects.filter(id__gt=5,id__lte=10)      #this month day2
place_list = Place.objects.filter(id__gt=15,id__lte=20)     #this month day3
....
place_list = Place.objects.filter(id__gt=80)                #this month day20

But how to make this thought to the result I want? Please teach me,Thank you.

Upvotes: 0

Views: 46

Answers (1)

Tianqi Tong
Tianqi Tong

Reputation: 156

Try Crontab: django-crontab github. It is a django app that uses linux's crontab to do scheduled jobs.

You may want to add a new column in you database as "processed",

and add a scheduled job as in the Example, set the run time like this:

CRONJOBS = [
    ('0 20 * * *', 'myproject.myapp.cron.my_scheduled_job'),
]

and 

python manage.py crontab add

by doing this, your code will run at 20:00 daily.

and in your cron.py

def my_scheduled_job():
    p = Place.objects.filter(processed=False)[:5]
    p.do_something()

to get the first 5 unprocessed elements and process them.

Upvotes: 1

Related Questions