CODEWITHSUNDEEP
web-scrapingbeautifulsouphtml-parsing
optimistic-zia
optimistic-zia

Reputation: 111

Unable to get a particular tag while using beautiful soup

I wanted to extract information from the stack overflow website, when i want to get the text part of questions I did:

import requests
from bs4 import BeautifulSoup
response=requests.get("https://stackoverflow.com/")
soup=BeautifulSoup(response.text,"html.parser",multi_valued_attributes=None)

for tag in soup.find_all('a',class_='question-hyperlink'):
    print(tag)

This gives no output at all.I think there is some problem when i filter the class but i am not sure what it is.

This one works fine:

import requests  
from bs4 import BeautifulSoup
response=requests.get("https://stackoverflow.com/questions")
soup=BeautifulSoup(response.text,"html.parser")
question=soup.select(".question-summary")

for a in question:
    print(a.select_one(".question-hyperlink").getText())

but what is wrong with the former one?

Upvotes: 0

Views: 77

Answers (1)

baduker
baduker

Reputation: 20042

You're missing questions in the url on this line in the frist code snippet:

response=requests.get("https://stackoverflow.com/")

This works fine:

import requests
from bs4 import BeautifulSoup

response = requests.get("https://stackoverflow.com/questions")
soup = BeautifulSoup(response.text, "html.parser")

for tag in soup.find_all('a', class_='question-hyperlink'):
    print(tag.getText(strip=True))

Output:

Pass a json object in function as a variable
iPhone Application Development in Windows 10 Platform
Jetty Websocket API Session
Exit from a multiprocessing Pool for loop using apply_async and terminate
bootstrap 5 grid layout col-md-6 not working correctly
R comparison (1) is possible only for atomic and list types
NeutralinoJS: error: missing required argument 'name'
Formatting text editor with Elementor

and so on ...

Otherwise, there's no such class of the anchor tag.

Upvotes: 1

Related Questions