CODEWITHSUNDEEP
pythonscrapy
Caliche Orozco
Caliche Orozco

Reputation: 11

Scrapy: extract text from span without id

I'm doing a spider for this website: link. I'm trying to get the price, but I couldn't this get me back none, I can get the title but I couldn't with the prices because the span haven't class, I don't know why get me back none because in the browser the xpath works.

imgCss = response.xpath("(//img[contains(@class, 'vtex-product-summary-2-x-imageNormal')]/@src)[2]").get()
title = response.xpath("(//article)[3]//span[contains(@class, 'vtex-product-summary-2-x-productBrand')]/text()").get()
discount = response.xpath("(//article)[3]//span[contains(@class, 'currencyContainer--summary txt-price-responsive')]//text()").get()
price = response.xpath("(//article)[3]//span[contains(@class, 'currencyContainer--summary t-heading-2-s')]//text()").get()

Response image:

Upvotes: 0

Views: 95

Answers (1)

msenior_
msenior_

Reputation: 2110

The information you seek is available in the page source. You can parse it as json and extract the information you need. You can start with the code below and extract the relevant information from the dictionary.

import scrapy
import json

class TestSpider(scrapy.Spider):
    name = 'test'
    allowed_domains = ['elektra.mx']

    def start_requests(self):
        yield scrapy.Request(url="https://www.elektra.mx/telefonia/celulares")

    def parse(self, response):
        data = response.xpath("(//*[@type='application/ld+json'])[2]/text()").get()
        data_json = json.loads(data)

        for product in data_json.get("itemListElement"):
            yield product.get("item")

Upvotes: 2

Related Questions