CODEWITHSUNDEEP
pythonpython-3.xweb-scrapingpython-requests
Michael
Michael

Reputation: 199

Web scraping with python how to get to the text

I'm trying to get the text from a website but can't find a way do to it. How do I need to write it?

link="https://www.ynet.co.il/articles/0,7340,L-5553905,00.html"
response = requests.get(link)

soup = BeautifulSoup(response.text,'html.parser')
info = soup.find('div', attrs={'class':'text14'})
name = info.text.strip()
print(name)

This is how it looks: enter image description here

i'm getting none everytime

Upvotes: 4

Views: 188

Answers (2)

johnsnow06
johnsnow06

Reputation: 131

import requests
from bs4 import BeautifulSoup
import json
link="https://www.ynet.co.il/articles/0,7340,L-5553905,00.html" 
response = requests.get(link)
soup = BeautifulSoup(response.text,'html.parser') 
info = soup.findAll('script',attrs={'type':"application/ld+json"})[0].text.strip()
jsonDict = json.loads(info)
print(jsonDict['articleBody'])

The page seems to store all the article data in json in the <script> tag so try this code.

Upvotes: 2

Michael
Michael

Reputation: 199

The solution is :

info = soup.find('meta', attrs={'property':'og:description'})

It gave me the text i needed

Upvotes: 1

Related Questions