CODEWITHSUNDEEP
pythonseleniumweb-scraping
Randomb
Randomb

Reputation: 43

Scroll down google maps webpage

I am trying to collect reviews on a google map webpage, however i can't find a way to scroll down the page to get all the reviews.

I'm using python 3 and Selenium package,

I've found different ways to scroll down social media pages such as facebook or IG, but the code doesn't work on google map. Also, I tried to locate the end key of the body tag, didn't work neither.

I'll be happy if someone can help,

thank you in advance,

Upvotes: 2

Views: 2105

Answers (2)

Sebastien D
Sebastien D

Reputation: 4482

One way to scroll down with selenium can be to click on a particular element on the page. If you do the following...

reviews_divs = driver.find_elements_by_class_name('section-review')
reviews_divs[-1].click()

... it will click on the last review and therefore scroll to it. Good but not yet what you want since the page is not loading new results...

However if you click on the first div after the section-review div, new reviews are now properly loaded:

driver.find_element_by_class_name('section-loading').click()

EDIT

The code above will be effective once you have clicked on the "Show all reviews" when you land on the store page you want to test:

driver.find_element_by_css_selector("button[class*='__button-text'").click()

Upvotes: -1

EatSleepCode
EatSleepCode

Reputation: 462

You can use the execute_script from selenium to scroll to the bottom of the page.

browser = webdriver.Chrome('chromedriver.exe', options=self.browserProfile)
brownser.get('your url here')
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")

If this does not scroll on the correct area you can perform a click on the div containing the reviews and then perform the window.scrollTo

 review_box= lambda: self.browser.find_element_by_xpath("xpath to div")
 review_box().click()

Hope this helps! :)

Upvotes: 2

Related Questions