CODEWITHSUNDEEP
csspython-3.xxpathscrapy
too_many_questions
too_many_questions

Reputation: 553

Scrapy chain selector with different parents

I want to chain two selectors together that have different parents. The selector I'm using currently:

..css('td:nth-child(8) > span.cap.mtv > ::text')

Which yields:

<Selector xpath="descendant-or-self::td[count(preceding-sibling::*) = 7]/span[@class and contains(concat(' ', normalize-space(@class), ' '), ' cap ') and (@class and contains(concat(' ', normalize-space(@class), ' '), ' mtv '))]/*/text()" data='$725,000'>

The issue I have is that I also want the following:

..xpath('td[8]/div/text()')

Which yields:

<Selector xpath='td[8]/div/text()' data='UFA'>

Ultimately I want to use an item loader and extract to get:

$725,000 UFA ...

I want to achieve something similar to the following..

...xpath('td[8]').css('span.cap.mtv > ::text').xpath('/div/text()')

I have previously just sort of re-scraped an element w/ another set of selectors if the program had previously found nothing but would much rather have this sort of 'either/or' flexibility. Or would I be better of looking at another selector all together for this situation?

Any help is much appreciated!

Upvotes: 0

Views: 797

Answers (1)

stranac
stranac

Reputation: 28236

If you're using item loaders, you can simply add multiple selectors for a single field as shown in scrapy docs.

Something like this should work, after creating a loader:

loader.add_css('field', 'td:nth-child(8) > span.cap.mtv > ::text')
loader.add_xpath('field', 'td[8]/div/text()')

Your input/output processors would then be responsible for how this information is combined.

Upvotes: 1

Related Questions