CODEWITHSUNDEEP
pythonselenium-webdriverweb-scrapinggoogle-crawlers
Chris Ho
Chris Ho

Reputation: 27

Scrolling in the pop-up Google Play Store Selenium

I want to crawl all reviews of an app from Google Play Store.

I have successfully opened pop up window that contains all the reviews but cannot scroll down to load more reviews. I can get only 40 reviews overall. Here is my xpath: enter image description here

Here is my solution:

more_button = driver.find_element(by=By.XPATH, value="//span[contains(text(),'See all reviews')]")
more_button.click()
time.sleep(5)

# Scroll down
popup = driver.find_elements(by=By.XPATH, value="/html/body/div[4]/div[2]/div/div/div/div/div[2]/div")

# Find all the reviews and their details
reviews = driver.find_elements(by=By.XPATH, value="//div[@class='RHo1pe']")
driver.execute_script('arguments[0].scrollTop = arguments[0].scrollHeight', popup)

Can someone please help me to achieve all the hidden reviews in this app. Thank you so much!

Upvotes: 1

Views: 216

Answers (1)

Mehmet Celik
Mehmet Celik

Reputation: 51

You are trying to load reviews, but only a limited amount of reviews are loaded with each scroll. To get more reviews, you need to scroll down, wait for the reviews to load, and then scroll down again.

This called lazy loading, where a website loads content as the user scrolls down the page, rather than loading everything at once. To get more reviews, you'll need to continue scrolling down and waiting for the reviews to load before scrolling again.

Upvotes: 1

Related Questions