CODEWITHSUNDEEP
pythonweb-scrapingscrapy
saraherceg
saraherceg

Reputation: 341

Cannot extract class values

I have been trying to scrape information from this website: "https://www.privateproperty.co.za/for-sale/western-cape/cape-town/55"

Specifically, I have an issue with trying to obtain number of bedrooms, bathrooms and garages.

Right now, I get index out of the range error and I am not sure how to fix this. Any suggestions?

       for prop in response.css('div.resultsItemsContainer a'):
           link = 'https://www.privateproperty.co.za' + prop.css('::attr(href)').get()
           title = prop.css('div.title::text').get()
           price = prop.css('div.priceDescription::text').re(r'\d+')

           bedrooms, bathrooms, garages = None, None, None
           for i in range(len(prop.css('div.features.row div::attr(class)'))):
               counter = i - 1
               if prop.css('div.features.row div::attr(class)')[i].get() == 'icon bedroom':
                   bedrooms = prop.css('div.features.row div::text')[counter].get()
               elif prop.css('div.features.row div::attr(class)')[i].get() == 'icon bathroom':
                   bathrooms = prop.css('div.features.row div::text')[counter].get()
               elif prop.css('div.features.row div::attr(class)')[i].get() == 'icon garage':
                   garages = prop.css('div.features.row div::text')[counter].get()

Example error:

    garages = prop.css('div.features.row div::text')[counter].get()
  File "/opt/anaconda2/lib/python2.7/site-packages/parsel/selector.py", line 61, in __getitem__
    o = super(SelectorList, self).__getitem__(pos)
IndexError: list index out of range

Thank you!

Upvotes: 0

Views: 54

Answers (1)

gangabass
gangabass

Reputation: 10666

Pretty easy with XPath:

response.xpath('//div[@class="feature"][contains(., "Baths")]/span/text()').get()

Upvotes: 1

Related Questions