CODEWITHSUNDEEP
pythonregexxpathscrapy
hjschmid
hjschmid

Reputation: 29

Scrapy SgmlLinkExtractor how go get number inside span tag

How can I get the integer number highlighted at this specific location:

number inside span tag

I got the follwing XPath from Google Chrome:

//*[@id="page"]/main/div[4]/div[2]/div[1]/div/div/div[2]/div[4]/div/div[1]/span

So I definded the following XPath statement with srapy to retreive the number:

id = response.xpath('//*[@id="page"]/main/div[4]/div[2]/div[1]/div/div/div[2]/div[4]/div/div[1]/span').extract()

However the variable id remains empty, my spider doesn't seem to crawl any information. How should I rewrite the statement to have access to this specific element?

Upvotes: 0

Views: 104

Answers (1)

lauda
lauda

Reputation: 4173

As a rule, to avoid later debugging to have a stable run you need to avoid using absolute xpath's or any xpath that is not flexible on minor changes of the page structure.

From the information available in the picture, your xpath should be:

//*[@class='nr']/span

For basic overview of the xpath rule you can take a look on w3schools xpath selectors

Upvotes: 1

Related Questions