CODEWITHSUNDEEP
pythonscrapy
taga
taga

Reputation: 3895

Scrapy does not have command 'crawl'

I started to learn Scrapy but right away I get an error Unknown command: crawl. I do not know why im getting this, but in py Scrapy commands I do not have that command. Im using python 3.6 and pycharm as editor.

(venv) C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts>scrapy crawl quotes
Scrapy 1.7.3 - no active project

Unknown command: crawl

Use "scrapy" to see available commands

(venv) C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts>scrapy
Scrapy 1.7.3 - no active project

Usage:
  scrapy <command> [options] [args]

Available commands:
  bench         Run quick benchmark test
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

This is the basic code that I wrote:

import scrapy

class My_Scraper(scrapy.Spider):

    name = 'quotes'

    #url sajtova koje scrapujem
    start_urls = [
        'https://www.nekretnine.rs/'
        ]

    def parse(self, response):

        title = response.css('title').extract()

        yield {'title text': title}

Upvotes: 3

Views: 7667

Answers (3)

Andes Lam
Andes Lam

Reputation: 202

I know this is an old post but I would also like to contribute an additional instance that may result in the missing of crawl command

if you accidentally delete the cfg file, crawl command will disappear too.

Upvotes: 4

Arun Augustine
Arun Augustine

Reputation: 1766

You must get out from the script folder and move to the project directory where scrapy.cfg file placed.

Run the scrapy crawl from there it will work.

Upvotes: 8

reg202
reg202

Reputation: 160

You need to be inside the project folder within the Scrapy folder.

You are currently trying to run the command from C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts

but it should be something like C:\Users\Pc\PycharmProjects\web skreper\venv\Scripts\Scrapy\My_Scraper

Upvotes: 8

Related Questions