Reputation: 441
I want to use the Advanced Python Scheduler (https://apscheduler.readthedocs.io/en/stable/) in order to scrape popular times from Google every hour (40 grocery stores in list markets, from 6:00 to 18:00) on a specific day.
My code works if I start it manually every hour, but I don't know how to write correct code in order to start the AP Scheduler. It was running for quite some time, but the resulting data frame was empty.
I was looking into the manual and other questions, but I couldn't figure out how to write the code.
from apscheduler.schedulers.blocking import BlockingScheduler
def job():
for market in markets:
data = livepopulartimes.get_populartimes_by_address(market)
current_popularity_ = pd.DataFrame([[date.today().strftime("%Y-%m-%d"),
date.today().strftime("%A"),
datetime.now().strftime("%H:%M:%S"),
pd.DataFrame.from_dict(data, orient='index').loc['current_popularity'].values[0]
]],
columns=['date','day','time','value'])
current_popularity_['market'] = market
current_popularity = current_popularity.append(current_popularity_)
sched = BlockingScheduler()
sched.add_job(
job,
trigger='cron',
hour='06-18',
start_date = '2021-02-04',
end_date = '2021-02-04'
)
sched.start()
Upvotes: 0
Views: 282
Reputation: 441
So I corrected the dates in my code and tried to run it again. But again, it didn't work. The resulting data frame was empty. The code works if I start it manually (data frame is not empty). So if anyone has an idea what is wrong with the code for the scheduler, please let me know.
Upvotes: 1