CODEWITHSUNDEEP
pythonxpathscrapy
comiventor
comiventor

Reputation: 4122

Invalid expression Using scrapy parser to extract Xpath

I want to extract certain attribute value from a HTML response. The specific element I am interested in looks like below

<meta property="x:y:z" content="interesting">

I am using following code to extract "interesting" part. Its giving me Invalid expression. I even validated the expression using a third party tool to generate XPath.

import requests
from scrapy import Selector

req = requests.get(some_url, headers=user_agent)
hxs = Selector(req)
links = hxs.xpath('//meta[@property="x:y:z"]@content')

Any clues?

Upvotes: 0

Views: 154

Answers (1)

mehrdadep
mehrdadep

Reputation: 1019

use / before @content and extract() at the end:

import requests
from scrapy import Selector

req = requests.get(some_url, headers=user_agent)
hxs = Selector(req)
links = hxs.xpath('//meta[@property="x:y:z"]/@content').extract()
#result: ['interesting']

Upvotes: 1

Related Questions