CODEWITHSUNDEEP
pythonscrapyscheduled-tasks
Tribic
Tribic

Reputation: 108

Schedule Scrapy Spider via Batch with Win10 Task-Scheduler

I´m still pretty new to python, but I managed to get my spider to crawl the things I need.

Then I created a batch file, to run that spider. The batch looks like this:

call C:\Users\Tribic\Anaconda2\Scripts\activate.bat
call cd cmc\cmc\spiders\
call scrapy runspider cmc_job.py -o out3.csv

Line1 switches the cmd to python

Line2 goes to the corresponding path

Line3 runs the spider, which saves the data to an csv file.

This works fine, as long as I click the batch (mouse or keyboard) - but when scheduled with Win10 Task Scheduler, it does not work... The cmd windows opens for 1/2 sec, I think it does not wait for the first line to be executed, so line 3 is a bad command and then it closes.

Once again, it works fine when started manually, so I don´t get the difference between executing by hand and by system.

Thanks in advance :)

Upvotes: 2

Views: 1147

Answers (1)

matthieu.cham
matthieu.cham

Reputation: 511

Try the following syntax, it works for me in Windows task scheduler. Notice the quotes and the invokation of cmdline.py

@Echo Off
REM activate Python venv
CALL "C:\Users\Tribic\Anaconda2\Scripts\activate.bat"
CD "C:\cmc\cmc\spiders\"
CALL "C:\Users\Tribic\Anaconda2\Scripts\python.exe" "C:\Users\Tribic\Anaconda2\Scripts\Lib\site-packages\scrapy\cmdline.py" runspider cmc_job.py -o out3.csv
deactivate

Upvotes: 3

Related Questions