CODEWITHSUNDEEP
pythonxpathweb-scrapingscrapy
Gianluca
Gianluca

Reputation: 39

How to have an arbitrary value in an xpath command for web scraping with python scrapy

How can I define an xpath command in python (scrapy) to accept any number at the place indicated in the code. I have already tried to put an * or any() at the position.

table = response.xpath('//*[@id="olnof_**here I want to accept any value**_altlinesodd"]/tr[1]/TD[1]/A[1]')

Upvotes: 2

Views: 79

Answers (3)

ThunderMind
ThunderMind

Reputation: 799

Now assuming your any as like ANY, so you can try this. x_path = '//*[@id="olnof_{}_altlinesodd"]/tr[1]/TD[1]/A[1]' x_path.format("put your any here, may b from rand function or extracting the value from somewhere else.") Then, table = response.xpath(x_path) this will do the work.

Upvotes: 0

stranac
stranac

Reputation: 28236

You can do this using regular expressions:

table = response.xpath('//*[re:test(@id, "^olnof_.+_altlinesodd$")]/tr[1]/TD[1]/A[1]')

Upvotes: 2

Andersson
Andersson

Reputation: 52665

You can try below workaround:

'//*[starts-with(@id, "olnof_") and contains(@id, "_altlinesodd")]/tr[1]/TD[1]/A[1]'

ends-with(@id, "_altlinesodd") suites better in this case, but Scrapy doesn't support ends-with syntax, so contains used instead

Upvotes: -1

Related Questions