CODEWITHSUNDEEP
pythonhtmlpython-3.xweb-scrapingyoutube
Kab2k
Kab2k

Reputation: 301

Webscraping Youtube pages

I'm trying to web scrape a youtube channel name via a link. But I get the error code:

title = response.find_all('div', class_= "style-scope ytd-channel-name")
AttributeError: 'Response' object has no attribute 'find_all'

Link to site: https://www.youtube.com/channel/UCHOgE8XeaCjlgvH0t01fVZg

Code:

url = 'https://www.youtube.com/channel/UCHOgE8XeaCjlgvH0t01fVZg'
response = requests.get(url)

title = response.find_all('div', class_= "style-scope ytd-channel-name")
soup = BeautifulSoup(title.text, 'lxml')
print(soup)

Thank you!

Upvotes: 1

Views: 376

Answers (2)

Abhishek Rai
Abhishek Rai

Reputation: 2227

We can use this.

from requests_html import HTMLSession
from bs4 import BeautifulSoup as bs # importing BeautifulSoup


video_url = "https://www.youtube.com/channel/UCHOgE8XeaCjlgvH0t01fVZg"
# init an HTML Session
session = HTMLSession()
# get the html content
response = session.get(video_url)
# execute Java-script
response.html.render(sleep=1)
# create bs object to parse HTML
soup = bs(response.html.html, "html.parser")
name = soup.find('yt-formatted-string', class_='style-scope ytd-channel-name')
print(name.text)

Output:-

TheTekkitRealm

Upvotes: 1

AverageHomosapien
AverageHomosapien

Reputation: 708

The following code returns the divs:

url = "https://www.youtube.com/channel/UCHOgE8XeaCjlgvH0t01fVZg"
req = requests.get(url)
soup = BeautifulSoup(req.text, "html.parser")
print(soup.div)

The value being returned can be changed through the 'soup.' value (e.g. soup.title).

I link to the documentation because I think it would be useful for you to look at too: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#

Upvotes: 0

Related Questions