CODEWITHSUNDEEP
pythonbeautifulsouppython-requests
dark horse
dark horse

Reputation: 447

Extract data using Beautiful Soup and Requests

I am trying to scrape data from Stackoverflow using Beautiful soup and requests package in Python. I have been able to extract most of the details, however when I try to extract the reputation scores of an user I am only able to pull data for reputation score and Gold, but not able to extract data for Silver and Bronze counts.

Given below is the code I am using to extract:

from bs4 import BeautifulSoup
import requests

source = requests.get('https://stackoverflow.com/questions/53968946/how-can-i-limit-function-slot-play-just-for-5-turn-with-do-while-loop').text
soup = BeautifulSoup(source,'lxml')
article = soup.find('div', class_='inner-content clearfix')
user_reputation_score = article.find('span', class_='reputation-score').text
print(user_reputation_score)

Code for Gold badge:

gold_badge = article.find('div', class_='-flair').find('span', class_='badgecount').text
print(gold_badge)

Wondering how to extend the above to extract data for Silver and Bronze batches.

Use the below link to test:

https://stackoverflow.com/questions/53968946/how-can-i-limit-function-slot-play-just-for-5-turn-with-do-while-loop

Please note I am doing this for pure educational purpose. Thanks.

Upvotes: 1

Views: 101

Answers (1)

ewwink
ewwink

Reputation: 19184

find() return first element, to get multiple element use find_all()

badge = article.find('div', class_='-flair').find_all('span', class_='badgecount')
gold_badge = badge[0].text
silver_badge = badge[1].text
bronze_badge = badge[2].text
print(gold_badge, silver_badge, bronze_badge) # 2 7 26

Upvotes: 1

Related Questions