CODEWITHSUNDEEP
pythonseleniumweb-scrapingbeautifulsoup
xxx
xxx

Reputation: 203

Unable to scraping a review page using python,selenium and beautiful soup

I'm trying to scrape a Flipkart review page

https://www.flipkart.com/reviews/b9ec5c5c-3814-40ef-8f73-84f253ee09ce

I'm using python, beautiful soup & selenium. Below is code

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(executable_path='D:\ESS\Intern-Shubham\chromedriver.exe',options=options)
driver.get('https://www.flipkart.com/reviews/b9ec5c5c-3814-40ef-8f73-84f253ee09ce')
time.sleep(20)
WebDriverWait(driver,
delay).until(EC.presence_of_element_located(driver.find_elements_by_class_name('_3t4Eas')))
print('Page is ready!')

No matter how much delay I'm adding the actual review is not been loaded into the page, below is the snip of review which loads up after some time. enter image description here

Any hints suggestion would be helpful.

Upvotes: 1

Views: 98

Answers (1)

PDHide
PDHide

Reputation: 19989

import By.

from selenium.webdriver.common.by import By

WebDriverWait(driver,
              20).until(EC.presence_of_element_located((By.CLASS_NAME,'_3t4Eas')))
print('Page is ready!')

your locator is wrong , now you can remove time.sleep()

you should pass a tuple with locator strategy and locator , not element

Upvotes: 1

Related Questions