CODEWITHSUNDEEP
pythonselenium-webdriverweb-scraping
Srushti Parab
Srushti Parab

Reputation: 31

selenium - webdriver won't go back to previous page

I am trying to click on a link, scrape data from that webpage, go back again, click on the next link and so on. But, I am not able to go back to the previous page for some reason. I observed that I can execute the code to go back if I am outside the loop, and I can't figure out what is wrong with the loop. I tried to use driver.back() too and yet it won't work. Any help is appreciated!! TYI

x = 0 #counter
contents=[]
for link in soup_level1.find_all('a', href=re.compile(r"^/new-homes/arizona/phoenix/"), tabindex=-1):

    python_button =driver.find_element_by_xpath("//div[@class='clearfix len-results-items len-view-list']//a[contains(@href,'/new-homes/arizona/phoenix/')]")

    driver.execute_script("arguments[0].click();",python_button)

    driver.implicitly_wait(50)

    soup_level2=BeautifulSoup(driver.page_source, 'lxml')

    a=soup_level2.find('ul',  class_ ='plan-info-lst')

    for names in a.find('li'):
         contents.append(names.span.next_sibling.strip())


    driver.execute_script("window.history.go(-1)")

    driver.implicitly_wait(50)

    x += 1

Upvotes: 1

Views: 5818

Answers (2)

undetected Selenium
undetected Selenium

Reputation: 193308

Some more information about your usecase interms of:

  • Selenium client version
  • WebDriver variant and version
  • Browser type and version

would have helped us to debug the issue in a better way.

However to go back to the previous page you can use either of the following solutions:

  • Using back(): Goes one step backward in the browser history.

    • Usage:

      driver.back()

  • Using execute_script(): Synchronously Executes JavaScript in the current window/frame.

    • Usage:

      driver.execute_script("window.history.go(-1)")

Usecase Internet Explorer

As per @james.h.evans.jr's comment in the discussion driver.navigate().back() blocks when back button triggers a javascript alert on the page if you are using at times back() may not work and is pretty much expected as navigates back in the history by using the COM GoBack() method of the IWebBrowser interface. Given that, if there are any modal dialogs that appear during the execution of the method, the method will block.

You may even face similar issues while invoking forward() in the history, and submitting forms. The GoBack method can be executed on a separate thread which would involve calling a few not-very-intuitive COM object marshaling functions e.g. CoGetInterfaceAndReleaseStream() and CoMarshalInterThreadInterfaceInStream() but there seems we can't do much about that.

Upvotes: 3

AzyCrw4282
AzyCrw4282

Reputation: 7744

Instead of using driver.execute_script("window.history.go(-1)")

You can try using

driver.back() see here

Please be aware that this functionality depends entirely on the underlying driver. It’s just possible that something unexpected may happen when you call these methods if you’re used to the behavior of one browser over another.

Upvotes: 0

Related Questions