CODEWITHSUNDEEP
pythondirectoryscrapy
tadm123
tadm123

Reputation: 8793

(Python) Cannot find any items in imported module scrapy

I started a project using scrapy on this path C:\Users\PATTY\Desktop\Project_Website\Scrapy

The problem is that most of my items imported are not found. For example in quotes.py it succesfully imports module scrapy but it can't open its items:

import scrapy


class QuotesSpider(scrapy.Spider):   # Error Cannot find reference Spider in imported module scrapy
    name = 'quotes'
    allowed_domains = ['quotes.toscrape.com/']          
    start_urls = ['http://quotes.toscrape.com/']

    def parse(self, response):
         pass

similarly in my items.py file:

import scrapy


class QuotesSpiderItem(scrapy.Item):   # Error Cannot find reference 'item' in imported module scrapy
    # define the fields for your item here like:
    # name = scrapy.Field()
    pass

Here's my project directory:

Scrapy>
  quotes_spider>
       quotes_spider>
             spiders>
                  __init__.py
                  quotes.py
             __init__.py
             items.py
             middlewares.py
             pipelines.py
             settings.py
       scrapy.cfg

Any help would really be appreciated.

Upvotes: 0

Views: 1055

Answers (2)

Dave S
Dave S

Reputation: 1009

It looks like you are working with the tutorial, but what you posted is incomplete which makes it hard to see what is going on in your case.

I would double check your code. If it looks correct, check your log output.

The log should look something like this:

[ ... Scrapy log here ... ]
2016-09-19 12:09:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET    http://quotes.toscrape.com/page/1/> (referer: None)
[s] Available Scrapy objects:
[s]   scrapy     scrapy module (contains scrapy.Request, scrapy.Selector, etc)
[s]   crawler    <scrapy.crawler.Crawler object at 0x7fa91d888c90>
[s]   item       {}
[s]   request    <GET http://quotes.toscrape.com/page/1/>
[s]   response   <200 http://quotes.toscrape.com/page/1/>
[s]   settings   <scrapy.settings.Settings object at 0x7fa91d888c10>

Check that your return code on the response is 200.

More information can be found here:

https://doc.scrapy.org/en/latest/intro/tutorial.html

Good luck!

Upvotes: 1

duwudi
duwudi

Reputation: 11

Your directory name Scrapy is conflicting with import scrapy in Pycharm

I had the same issue, simply change your top-level directory name

Upvotes: 1

Related Questions