CODEWITHSUNDEEP
pythonweb-scrapingscrapy
SSD93
SSD93

Reputation: 39

Can't make my first spider run,any advice?

This is my first time using scrapy and maybe the third in python, so i'm a noob. The problem with this code is that it doesn't even enter the page.

I have tried to use: scrapy shell 'https://www.zooplus.es/shop/tienda_perros/pienso_perros/pienso_hipoalergenico'

This works and then using...

response.xpath('//*[@class="product__varianttitle ui-text--small"]')

... I can retrieve information.

My code:

import scrapy

class ZooplusSpider(scrapy.Spider):
    name = 'Zooplus'
    allowed_domains = ['zooplus.es']
    start_urls = ['https://www.zooplus.es/shop/tienda_perros/pienso_perros/pienso_hipoalergenico']

def parse(self, response):
    item= scrapy.Item() 
    item['nombre']=response.xpath('//*[@class="product__varianttitle ui-text--small"]')             
    item['preciooriginal']=response.xpath('//*[@class="product__prices_col prices"]')
    item['preciorebaja']=response.xpath('//*[@class="product__specialprice__text"]')
    return item

The error message says:

2019-08-30 21:16:57 [scrapy.core.engine] INFO: Spider opened
2019-08-30 21:16:57 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2019-08-30 21:16:57 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2019-08-30 21:16:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.zooplus.es/robots.txt> (referer: None)
2019-08-30 21:16:57 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.zooplus.es/shop/tienda_perros/pienso_perros/pienso_hipoalergenico> from <GET https://www.zooplus.es/shop/tienda_perros/pienso_perros/pienso_hipoalergenico/>
2019-08-30 21:16:58 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.zooplus.es/shop/tienda_perros/pienso_perros/pienso_hipoalergenico> (referer: None)
2019-08-30 21:16:58 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.zooplus.es/shop/tienda_perros/pienso_perros/pienso_hipoalergenico> (referer: None)

Upvotes: 0

Views: 54

Answers (1)

Madhav Kumar
Madhav Kumar

Reputation: 169

I think you haven't defined the fields for your items.py the error is coming from item['nombre']

Either you should define the field in items.py or simply replace item= scrapy.Item() with item = dict()

Upvotes: 1

Related Questions