CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
komko
komko

Reputation: 112

How do I get text from div?

I want to scrape info from one website, to get information about user, for bot to have stored in a channel.

I've watched tutorials but none worked.

import requests
from bs4 import BeautifulSoup

r = requests.get("http://www.rubyrealms.com/user/KOMKO190/")
content = r.content
soup = BeautifulSoup(content, "html.parser")
print(soup)

How would I get the DIVs and the text from div?

Upvotes: 0

Views: 192

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195438

I parsed some sample items from the page, that should get you going started:

import requests
from bs4 import BeautifulSoup

r = requests.get("http://www.rubyrealms.com/user/KOMKO190/")
content = r.content
soup = BeautifulSoup(content, 'lxml')

name = soup.select_one('div.profile-title').text.strip()
about = soup.select_one('div.profile-about').text.strip()
achievements = [span['title'] for span in soup.select('div.achievements span[title]')]
stats = [div.text.strip() for div in soup.select('div.stats div')]

print('{: <20}{}'.format('Name:', name))
print('{: <20}{}'.format('About:', about))
print('{: <20}{}'.format('Achievements:', achievements))
print('{: <20}{}'.format('Stats:', stats))

Prints:

Name:               KOMKO190
About:              Hey! My name is KOMKO190, you maybe know me from the forums or discord. I am a programmer, I know a bit of JavaScript, small portion of C++, Python and html/css. Mostly python. My user ID is 7364. ||| 5th owner of Space Helmet :)
Achievements:       ['Verified', 'Novice Collector', 'XP Prodigy', '1k Forum Posts']
Stats:              ['297 Profile Views', '1826 Forum Posts', 'Last seen 40 mins ago', 'Joined Apr 27, 2019']

Upvotes: 1

Related Questions