CODEWITHSUNDEEP
pythonscrapy
fobus
fobus

Reputation: 2058

Python - Unable to import local library

I have a scrapy crawler, I wan to use a local library in my crawler.

So, here is my directory model :

enter image description here

There is two important files db/base.py and /crawler/spiders/adilisik.py

here is base.py

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

Base = declarative_base()

engine = create_engine("mysql+pymysql://xxx:yyy@localhost/test-db")
Session = sessionmaker(bind=engine)
session = Session()

here is some lines from adilisik.php

    # -*- coding: utf-8 -*-
    import hashlib
    import re
    import scrapy

    from crawler.db.base import Base

    class AdilisikSpider(scrapy.Spider):
        name = "adilisik"
        allowed_domains = ['adl.com.tr']
        start_urls = ['http://adl.com.tr']
        urls = set()


        def __init__(self, retailer='', *args, **kwargs):
            super(AdilisikSpider, self).__init__(*args, **kwargs)

        def parse(self, response):
.....
.....

But I'm not able make this code work.

this line breaks my code.

from crawler.db.base import Base

I'm getting this error : 

    from crawler.db.base import Base
ImportError: No module named 'crawler.db'
Could not load spiders from module 'crawler.spiders'. Check SPIDER_MODULES setting

What I am doing wrong?

Edit 1:

After Moinuddin Quadri's suggestion I have created init.py in crawler directory and renamed the crawler directory. But now I'm getting the error below

ImportError: No module named 'crawler.settings'

Upvotes: 1

Views: 480

Answers (1)

Moinuddin Quadri
Moinuddin Quadri

Reputation: 48090

__init__.py is missing in your crawler directory. Add an empty __init.__.py and then you will be able to import crawler.db module.

Also, note that you have two directories as crawler (+ one more which is your project). Rename one of the directory else you might be facing more errors related to import.

Upvotes: 1

Related Questions