CODEWITHSUNDEEP
pythonpython-2.7parsingxpathscrapy
SpanishBoy
SpanishBoy

Reputation: 2225

Extract Outer Text with Scrappy

I need to parse following fragment:

<span>    Lekhwiya&nbsp;v&nbsp;<strong class="winner-strong">Zobahan</strong></span>

or

<span>    <strong class="winner-strong">Sepahan</strong>&nbsp;v&nbsp;Al&nbsp;Nasr&nbsp;(UAE)</span>

as Lekhwiya v Zobahan and <Sepahan v Al' Nasr'(UAE) properly.

I was trying to parse as:

team_1 = block.xpath('.//span/text()').extract()[:2]
team_1 = team_1[0].strip() + team_1[1].strip() 
team_2 = block.xpath('.//span/strong/text()').extract()[0]

item['match'] = team_2.strip() + ' ' + team_1 if team_1[0] == 'v' else team_1 + ' ' + team_2.strip()

As for me, it's ugly solution. What is the best approach to do it?

Upvotes: 1

Views: 352

Answers (1)

paul trmbrth
paul trmbrth

Reputation: 20748

You can use XPath's string() function, or normalize-space() even:

In [1]: text = '''
   ...: <span>    Lekhwiya&nbsp;v&nbsp;<strong class="winner-strong">Zobahan</strong></span>
   ...: <span>    <strong class="winner-strong">Sepahan</strong>&nbsp;v&nbsp;Al&nbsp;Nasr&nbsp;(UAE)</span>
   ...: '''

In [2]: import scrapy

In [3]: selector = scrapy.Selector(text=text, type="html")

In [4]: for span in selector.xpath('//span'):
   ...:     print(span.xpath('string(.)').extract_first())
   ...:     
    Lekhwiya v Zobahan
    Sepahan v Al Nasr (UAE)

In [5]: for span in selector.xpath('//span'):
    print(span.xpath('normalize-space(.)').extract_first())
   ...:     
Lekhwiya v Zobahan
Sepahan v Al Nasr (UAE)

Upvotes: 2

Related Questions