CODEWITHSUNDEEP
htmlscrapy
dedepene
dedepene

Reputation: 23

How to grab text inside a script tag with scrapy?

I need to grab as text the contents of a script tag with a very specific attribute with the scrapy library. Essentialy the BeautifulSoup equivalent of this:

js_content = soup.find("script",type="application/ld+json").get_text()

I tried this, but the result is not quite what I need.

response.css('script').attrib['type']

Upvotes: 0

Views: 59

Answers (1)

SuperUser
SuperUser

Reputation: 4822

CSS:

response.css('script[type="application/ld+json"]::text').get()

xpath:

response.xpath('//script[@type="application/ld+json"]/text()').get()

Basically we're finding a script tag that has an attribute type with a value of application/ld+json and grabbing the text.

Upvotes: 4

Related Questions