CODEWITHSUNDEEP
pythonparsingbeautifulsoup
egorkh
egorkh

Reputation: 488

BeautifulSoup doesn't shows all the elements in tag

I'm trying to parse this website and get information about auto in content-box card__body with BeautifulSoup.find but it doesn't find all classes. I also tried webdriver.PhantomJS(), but it also showed nothing.

Here is my code:

from bs4 import BeautifulSoup
from selenium import webdriver

url='http://www.autobody.ru/catalog/10230/217881/'

browser = webdriver.PhantomJS()
browser.get(url)
html = browser.page_source

soup = BeautifulSoup(html, 'html5lib') 
JTitems = soup.find("div", attrs={"class":"content-box__strong red-text card__price"})
JTitems

or 

w = soup.find("div", attrs={"class":"content-box card__body"})
w

Why doesn't this approach work? What should I do to get all the information about auto? I am using Python 2.7.

Upvotes: 0

Views: 1323

Answers (1)

MD. Khairul Basar
MD. Khairul Basar

Reputation: 5110

Find the table where your necessary info are. Then find all the tr and loop through them to get the texts.

from bs4 import BeautifulSoup
from selenium import webdriver

url='http://www.autobody.ru/catalog/10230/217881/'

browser = webdriver.Chrome()
browser.get(url)
html = browser.page_source

soup = BeautifulSoup(html, 'lxml')

price = soup.find('div', class_='price').get_text()
print(price)

for tr in soup.find('table', class_='tech').find_all('tr'):
    print(tr.get_text())

browser.close()
browser.quit()

Upvotes: 1

Related Questions