CODEWITHSUNDEEP
pythonpython-3.xselenium-webdriverweb-scrapinggeckodriver
Med Amin Hbaieb
Med Amin Hbaieb

Reputation: 3

How to collect data python using selenium geckodriver

I tried to scrap data from website using selenium firefox and geckodriver. I want to navigate from one page to another and extract informations. I am unable to move from current page to another, and force driver to back to the original page and move to next element in the list. I created the code bellow which click on first element and go to the specific page to collect data. Thank you

   binary = FirefoxBinary('/usr/bin/firefox')
    driver = webdriver.Firefox(firefox_binary=binary, executable_path=r'/home/twitter/geckodriver')
    try:
        driver.get('https://www..........')  
        list_element = driver.find_elements_by_xpath("//span[@class='icon icon-email']")
for element in list_element :         
        x = driver.current_url
        element.click()
        time.sleep(5)
        for ex in driver.find_elements_by_xpath('//span[@class = "valeur"]'):
            print (ex.text)
        driver.back()
except Exception as e:
         print (e)
driver.quit()

Upvotes: 0

Views: 200

Answers (1)

Abdullah Ahmed Ghaznavi
Abdullah Ahmed Ghaznavi

Reputation: 2099

This might happens because your driver/browser didn't get the new page (current page). add one line after element.click() or time.sleep(5) :

driver.switch_to.window(driver.current_window_handle)

then try to run your code again. Hope this helps you! :)

Upvotes: 2

Related Questions