CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
finnew
finnew

Reputation: 13

Beautiful soup gives output as "Searching for your content..." when trying to get main article text

I'm trying to extract the main article text from some pages on newswire website using beautiful soup. But instead of the text output. I get an output that says "Searching for your content..."

I would highly appreciate any help. I seem to be doing something fundamentally wrong here. (I'm sorry about that, if I'm doing it the wrong way as I'm a beginner in coding)

from bs4 import BeautifulSoup
from urllib.request import Request, urlopen

site= "https://www.newswire.ca/news-releases/rbc-capital-markets-launches-aiden-r-a-new-ai-powered-electronic-trading-platform-891623791.html"
hdr = {'User-Agent': 'Mozilla/5.0'}
req = Request(site,headers=hdr)
page = urlopen(req)
soup = BeautifulSoup(page)

text = soup.find_all('p', {"class":"text-muted"})[0].text.strip()
print(text)

Upvotes: 1

Views: 61

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195448

Try CSS selector ".release-body":

from textwrap import wrap
from bs4 import BeautifulSoup
from urllib.request import Request, urlopen

site = "https://www.newswire.ca/news-releases/rbc-capital-markets-launches-aiden-r-a-new-ai-powered-electronic-trading-platform-891623791.html"
hdr = {"User-Agent": "Mozilla/5.0"}
req = Request(site, headers=hdr)
page = urlopen(req)
soup = BeautifulSoup(page, "html.parser")
text = soup.select_one(".release-body").get_text(strip=True, separator=" ")

print("\n".join(wrap(text, width=80)))

Prints:

Traders and AI scientists at RBC and Borealis AI collaborate to deliver a real-
world AI solution to help improve trading results and insights for clients in a
measurable and explainable way NEW YORK , Oct. 14, 2020 /CNW/ - RBC Capital
Markets today announced the launch of Aiden ® , an AI-based electronic trading
platform that uses the computational power of deep reinforcement learning in its
pursuit of improved trading results and insights for clients. The Aiden ®
platform was developed jointly by RBC Capital Markets and Borealis AI, a world-
class AI research center created by RBC, as traders and AI scientists worked
side-by-side to create the initial bold concept and deliver a real-world
solution. In doing so, both organizations undertook one of the biggest
challenges in the field of AI today – applying deep reinforcement learning into
a constantly changing environment like equities trading, with measurable and
explainable results for its users. "Aiden ® speaks to our long-term commitment
to leverage next-generation AI technology to create differentiated solutions for
our clients," Bobby Grubert , Co-Head of Global Equities‎, RBC Capital Markets,
said. "Aiden ® is the result of a massive collaborative effort between RBC

...and so on.

Upvotes: 1

Related Questions