CODEWITHSUNDEEP
pythonbeautifulsoup
koyamashinji
koyamashinji

Reputation: 645

BeautifulSoup.select classname not working

I am trying to find tags by CSS class, using BeautifulSoup. Read documentation and tried different ways but the below code returns new_elem : []. Could you help me understand what I am doing wrong? Thanks.

import requests
from bs4 import BeautifulSoup

url = "https://solanamonkeysclub.com/#/#mint"

response = requests.get(url)
response.encoding = response.apparent_encoding
soup = BeautifulSoup(response.text, 'html.parser')
new_elems = str(soup.select('.ant-card-body'))
print(f'{"new_elem":10} : {new_elems}')

Upvotes: 1

Views: 52

Answers (2)

Md. Fazlul Hoque
Md. Fazlul Hoque

Reputation: 16187

As the url is dynamic,I use selenium with bs4 and getting the follwing output:

Code:

import requests
from bs4 import BeautifulSoup
import time
from selenium import webdriver


driver = webdriver.Chrome('chromedriver.exe')
url = "https://solanamonkeysclub.com/#/#mint"
driver.get(url)
time.sleep(8)

soup = BeautifulSoup(driver.page_source, 'html.parser')
new_elems = soup.select('.ant-card-body')
for new_elem in new_elems:
    print(f'{"new_elem":10} : {new_elem.text}')

OUTPUT:

new_elem   : 0
new_elem   : 0

Upvotes: 2

Tim Roberts
Tim Roberts

Reputation: 54955

Have you looked at the output at all? You should bring up this page in a browser and do a "view source", or do print(response.text) after you fetch it. The page as delivered contains no HTML elements. The entire page is built dynamically using Javascript.

You will need do use something like Selenium to scrape it.

Upvotes: 1

Related Questions