CODEWITHSUNDEEP
pythonxpathweb-scrapingscrapyweb-crawler
saraherceg
saraherceg

Reputation: 341

Get images using xpath

I have been trying to scrape information from this website https://www.fineandcountry.com/sa/property-for-sale/cape-town-clifton/property/929703 and I am having issues with getting all the images of the property: they are inside attribute style which gives me some of a struggle. What I have been trying to do:

images = response.xpath("//div[@class='search-results-gallery-property']
/a[@class='rotator_thumbs']/@style").extract()

But so far this is empty.

This is how it looks like:

<div class="search-results-gallery-property">
  <a style="background-image: 
url(https://static.propertylogic.net/property/8/200673/IMG_200673_3_small.jpg);" class="rotator_thumbs">
  </a></div>

Any suggestions on what I am doing wrong/ how to extract from attribute style? Thank you!

Upvotes: 1

Views: 455

Answers (2)

gangabass
gangabass

Reputation: 10666

By the way you can use .re() to parse each URL (using SIM code):

for item in response.xpath('//a[@data-fancybox-group="property-images"]/*[starts-with(@style,"background")]/@style').re(r"url\('([^']+)"):
    yield {"image_link":item}

Upvotes: 0

SIM
SIM

Reputation: 22440

The class name you tried with seems to have generated dynamically. This is how they are in the page source:

<a href="https://static.propertylogic.net/properties/8/972/1900/929703/IMG_yPAH7LUOOusZs642oS1TjGTl9WIXaOUKWWMsXiSnL0luoixXTAyNV32n9kiM_hd.jpeg" class="fancybox-thumbs col-md-4 col-sm-6 col-xs-12" data-fancybox-group="property-images" style="margin-bottom: 10px; padding-right: 5px; padding-left: 5px;">
    <div class="col-md-12" style="background: url('https://static.propertylogic.net/properties/8/972/1900/929703/IMG_yPAH7LUOOusZs642oS1TjGTl9WIXaOUKWWMsXiSnL0luoixXTAyNV32n9kiM_small.jpeg'); height: 160px; padding-right: 0px; padding-left: 0px; background-size: cover; background-position: center;"> </div>
</a>

You can go with either of the two to get the crude image links:

for item in response.css('a[data-fancybox-group="property-images"] > [style^="background"]::attr(style)').getall():
    yield {"image_link":item}

for item in response.xpath('//a[@data-fancybox-group="property-images"]/*[starts-with(@style,"background")]/@style').getall():
    yield {"image_link":item}

Upvotes: 1

Related Questions