CODEWITHSUNDEEP
shellxpathscrapy
Jackknife
Jackknife

Reputation: 105

How to extract the value of an HTML attribute using scrapy response.xpath?

I'm trying to extract the value of the attribute data-asin-price inside a <div> tag

Which in the example below you can see is 22.63

<div id="cerberus-data-metrics" style="display: none;" data-asin="B079GMRZ8S" data-asin-price="22.63" data-asin-shipping="0.0" data-asin-currency-code="AUD" data-substitute-count="-1" data-device-type="WEB" data-display-code="Asin is not eligible because it is not enabled"></div>

Is there any way to do this using response.xpath() with scrapy?

Thank you

Upvotes: 2

Views: 3718

Answers (2)

thehale
thehale

Reputation: 1776

In the current version of scrapy (v2.8), you can also use its built-in extensions to CSS selectors. These extensions may also be available in earlier version of scrapy.

response.css("div::attr(data-asin-price)").get()

In its generic form, replace CSS_SELECTOR and ATTRIBUTE_NAME as needed.

response.css("CSS_SELECTOR::attr(ATTRIBUTE_NAME)").get()

Upvotes: 0

Jackknife
Jackknife

Reputation: 105

I just wanted to post the answer I found.

To get the 22.63 value our of the data-asin-price attribute in scrapy shell I did the following:

response.xpath('//div[@id = "cerberus-data-metrics"]/@data-asin-price').extract_first()

Cheers

Upvotes: 3

Related Questions