Failing at importing modules on items.py file - Scrapy

Question

I'm new in Python and I'm currently learning webscraping with spiders. Following the tutorial, I stucked at relative importing with Python.

This is the structure of my current folder (provided by scrapy startproject p1):

My items.py file:

# Define here the models for your scraped items
#
# See documentation in:
# https://docs.scrapy.org/en/latest/topics/items.html

import scrapy


class Test(scrapy.Item):
    # define the fields for your item here like:
    # name = scrapy.Field()
    title = scrapy.Field()
    price = scrapy.Field()
    pass

In my filetwo.py, it contains:

import scrapy
from p1.items import Test

When I run the code, I get "ModuleNotFoundError: No module named 'p1'"

I also read some people online that faced the same problem, so I tried ..items import Test and still didn't work. It gave me the error: ImportError: attempted relative import with no known parent package

Can someone give me a light?

Thanks in advance!

Answer 1

The answer you accepted is wrong unfortunately. You should not be messing with the sys.path. You should instead:

$ cd P1
$ python -m p1.spiders.filetwo # note no .py

You are obviously running the filetwo.py script directly from inside the spiders package - that's an antipattern in python leading to all these errors. Messing with the sys.path on the other hand can lead to a slew of subtle bugs.

Answer 2

I am sure my comment should help you. But here is an example