CODEWITHSUNDEEP
pythonweb-scrapingscrapy
Rafał
Rafał

Reputation: 53

Get text from second child in DIV using scrapy, python

I'm trying to recive some text values from webiste and a small problem appeared.

Website with real estate data has a few features I'm trying to recive. I didn't have a problem with 'main' features like price. The code I'm using is as follows.

def get_offer_details(self, response):
    
    offer_item = ItemLoader(item=estateItem(), selector=response)

    offer_item.add_xpath('tittle', "//h1[@class='css-11t1qm5']/text()")
    offer_item.add_xpath('price', '//strong[@class="css-1mojccp"]/text()')

    yield offer_item.load_item()

I can use 'class' selector in examples above.

How Can I obtain the text value (in this example "2") from second div in this structure? There are few features with exactly the same structure, the only difference is the aria-label (bedrooms, market, etc..) so I cannot use 'class' selector.

<div role="region" aria-label="bedrooms" class="css-11ic80g">
    <div title="bedrooms" class="css-152vbi8">bedrooms<!-- -->:</div>
    <div title="2" class="css-1s5nyln">2</div>
</div>

This is how it looks like:

enter image description here

#I dont know.. maybe something like this? But it doesnt work..
offer_item.add_xpath('bedrooms', "//div[@aria-label='bedrooms'][1]/text()")

Thanks in advance.

Upvotes: 1

Views: 467

Answers (2)

gangabass
gangabass

Reputation: 10666

Another approach using following-sibling:::

offer_item.add_xpath('bedrooms', "//div[@title='bedrooms']/following-sibling::div[1]/text()")

Upvotes: 1

Parolla
Parolla

Reputation: 407

Try this XPATH to get text from first child:

"//div[@aria-label='bedrooms']/div[1]/text()"

or

"//div[@aria-label='bedrooms']/div[2]/text()"

for text from second

Upvotes: 1

Related Questions