CODEWITHSUNDEEP
pythonhtmlxpathscrapy
user5616520
user5616520

Reputation: 91

No xpath results in scrapy

I'm using this code. The last two Values are there like that because I was testing to see if either one of them will work- they don't, though.

def parse_again(self, response):
    sel = Selector(response)
    meta = sel.xpath('//div[@class="LWimg"]')
    items = []
    for m in meta:
        item = PageItem()
        item['link'] = response.url
        item['Stake'] = m.select('//div[@class="stakedLW"]/h1/text()').extract()
        item['Value'] = m.select('//p[@class="value"]/text()').extract()
        item['Value'] = m.select('//div[@class="value"]/span/span/text()').extract()
        items.append(item)
    return items

to retrieve data from this html source code

   <div class="LWimg">               
            <div class="stakedLW">              
                <span class="title">Stake</span>
                <span class="value">5.00</span>
                <span class="currency">&#128;</span>

My items.py looks like this

from scrapy.item import Item, Field

class Page(Item):
    Stake = Field()
    Value = Field()

The problem is that data is not retrieved, i.e. nothing is saved into a .csv in the end.

Any input is welcome.

Upvotes: 1

Views: 226

Answers (1)

eLRuLL
eLRuLL

Reputation: 18799

You are populating the Value field twice, so just the last one will work, and I think the correct way should be:

item['Value'] = response.xpath('//div[@class="stakedLW"]//span[@class="value"]/text()').extract_first()

The other fields are not necessary, just the link one.

Upvotes: 2

Related Questions