CODEWITHSUNDEEP
pythonselenium
abhanan93
abhanan93

Reputation: 249

Selenium Python: How to wait for a page to load after a click?

I want to grab the page source of the page after I make a click. And then go back using browser.back() function. But Selenium doesn't let the page fully load after the click and the content which is generated by JavaScript isn't being included in the page source of that page.

element[i].click()
#Need to wait here until the content is fully generated by JS.
#And then grab the page source.
scoreCardHTML  = browser.page_source
browser.back()

Upvotes: 16

Views: 40548

Answers (5)

phpjunkie
phpjunkie

Reputation: 195

Load the class for the browser your using and use implicitly_wait. implicitly_wait will wait for whatever element your trying to get to be found. This works great when you end up on a new page form the site you're parsing.

from datetime import datetime

from selenium.webdriver import Firefox
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException

firefox = Firefox()
firefox.implicitly_wait(time_to_wait = 120)  # in seconds: 2 minutes

firefox.get('https://www.google.com/')

timestamp = datetime.now().timestamp()
try:
    firefox.find_element(by = By.LINK_TEXT, value = 'aedgsf')
except NoSuchElementException:
    pass
print(datetime.fromtimestamp(datetime.now().timestamp() - timestamp).strftime('%M:%S:%f'))

>>> 02:00:910277

Upvotes: 0

varinder singh
varinder singh

Reputation: 41

you can also use seleniums staleness_of

from selenium.webdriver.support.expected_conditions import staleness_of

def wait_for_page_load(browser, timeout=30):
    old_page = browser.find_element_by_tag_name('html')
    yield
    WebDriverWait(browser, timeout).until(
        staleness_of(old_page)
    )

Upvotes: 4

Reza latifi
Reza latifi

Reputation: 1

Assuming "pass" is an element in the current page and won't be present at the target page. I mostly use Id of the link I am going to click on. because it is rarely present at the target page.

while True:
  try:
    browser.find_element_by_id("pass")
  except:
    break

Upvotes: -1

ahmed abdelmalek
ahmed abdelmalek

Reputation: 366

You can do it using this method of a loop of try and wait, an easy to implement method 

from selenium import webdriver
browser = webdriver.Firefox()
browser.get("url")
Button=''
while not Button:
    try:
        Button=browser.find_element_by_name('NAME OF ELEMENT')
        Button.click()
    except:continue

Upvotes: 3

wisnia
wisnia

Reputation: 656

As Alan mentioned - you can wait for some element to be loaded. Below is an example code

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.Firefox()
element = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, "element_id")))

Upvotes: 15

Related Questions