How to click and download with python selenium

Question

I try to download CSV data from GoogleTrend by selenium(python).

In previous, I tried to print source page and extract data that I want later. It worked for some period, but now it does not work.

I try to click download button to got CSV file but nothing happen. Do you have any idea for this case?

• I got button path from firebug+firepath (firefox plugin).

  html/body/div[2]/div[2]/div/md-content/div/div/div[1]/trends-widget/ng-include/widget/div/div/div/widget-actions/div/button[1]

• I try on chrome driver and firefox drive.

• This code; put 1 (word)argument that you want to get trend of search.

  import sys import time

  from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.action_chains import ActionChains from selenium.webdriver.support.ui import Select from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC

  def run_text_extract(search_word):

          try:
                       print(search_word)
  
  
                       driver = webdriver.Firefox('/home/noah/Desktop/Google_Trend_downloader/chromedriver/geckodriver')
           #            driver = webdriver.Chrome('/home/noah/Desktop/Google_Trend_downloader/chromedriver')
  
                       driver.get("https://trends.google.com/trends/explore?date=all&geo=TH&q="+search_word)
                       driver.find_element_by_xpath('html/body/div[2]/div[2]/div/md-content/div/div/div[1]/trends-widget/ng-include/widget/div/div/div/widget-actions/div/button[1]').click()
  
  
  
                       try: 
  
                            driver.manage().deleteAllCookies()
                            clear_cache(driver)
  
                       except TimeoutException as ex:
                              isrunning = 0
                              print("Exception has been thrown. " + str(ex))
                              print("Timeout line is", line ,".")
                              driver.close()
  
                       except Exception:
                              print ("Here 5")
                              pass
  
                       time.sleep(2) 
                       driver.close()
                       print("======== END_OF_FILE ===============")
  
  
          except:
                 pass
  

  if name == 'main': run_text_extract(sys.argv[1]) time.sleep(8) # run_text_extract()

Answer 1

I have navigated to the link you have provided. If you search for any term, you can see download csv button link will appear at the right side. But there will be 3 download csv buttton links with the same class or css selector are present. So you need to collect all the elements and loop through it so that you can click on specific element. In your case, I assume you want to click on first element. so below code should work. If you want 2nd or 3rd element to click change the index accordingly.

def run_text_extract(search_word):
    from selenium import webdriver
    from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
    import time
    profile = webdriver.FirefoxProfile()
    profile.set_preference("browser.download.folderList", 2)
    profile.set_preference("browser.download.manager.showWhenStarting", False)
    profile.set_preference("browser.download.dir", 'C:\\Python27')
    profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv")
    driver = webdriver.Firefox(firefox_profile=profile,executable_path=r'C:\\Python27\\geckodriver.exe')
    driver.get("https://trends.google.com/trends/explore?date=all&geo=TH&q="+ search_word)
    time.sleep(7)
    lst = driver.find_elements_by_css_selector(".widget-actions-item.export")
    lst[0].click()

run_text_extract("selenium")