CODEWITHSUNDEEP
seleniumgoogle-chromeweb-scrapingbeautifulsoupselenium-chromedriver
nate
nate

Reputation: 522

Webscraping issue with Python and Selenium

I am attempting to extract scroll-down menu data from here

I run the following script LINE-BY-LINE in a console and it runs with this error - there is other work of this online but I tried to use those resources with no avail:

import os
import time
import tempfile
from time import sleep
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

###
# downloadDir = tempfile.mkdtemp()

###
# prefs = { 'download.default_directory' : downloadDir }
# chromeOptions = webdriver.ChromeOptions()
# chromeOptions.add_experimental_option("prefs",prefs)

###
chrome_path = r'C:\Users\Nate\PycharmProjects\testing_1\chromedriver.exe'
cdriver_1  = webdriver.Chrome(chrome_path)
cdriver_1.get(url = 'http://webapps.rrc.texas.gov/eds/eds_searchUic.xhtml')

###

cdriver_1.find_element_by_xpath("//*[@id='SearchUicForm:district_1']/..").click()
# cdriver_1.find_element_by_xpath("//*[@id='SearchUicForm:county_1']").click()
# cdriver_1.find_element_by_xpath("//*[@id='ui-datepicker-div']/table/tbody/tr[1]/td[1]/a").click()
# cdriver_1.find_element_by_xpath("//*[@id='ui-datepicker-div']/table/tbody/tr[2]/td[2]").click()

ERROR:

selenium.common.exceptions.ElementNotVisibleException: Message: element not interactable
  (Session info: chrome=76.0.3809.132)
  (Driver info: chromedriver=72.0.3626.7 (efcef9a3ecda02b2132af215116a03852d08b9cb),platform=Mac OS X 10.14.3 x86_64)

Not that I want to grab any option from the scroll-down menu options like District, Permit Number, etc. How do I fix this?

Upvotes: 1

Views: 77

Answers (1)

JeffC
JeffC

Reputation: 25731

The issue is that you are clicking an LI element that is not currently visible. You must first click the dropdown to open it, then click the desired LI.

cdriver_1.find_element_by_id("SearchUicForm:district_label").click() # open the District dropdown
cdriver_1.find_element_by_xpath("//ul[@id='SearchUicForm:district_items']/li[.='01']").click() # select district "01"

I would suggest that you wrap this up in a function, set_district(district), and put the parameter into the locator so that it's reusable.

cdriver_1.find_element_by_xpath("//ul[@id='SearchUicForm:district_items']/li[.='{0}']".format(district)).click()

See How do I put a variable inside a string? for more info, examples.

Upvotes: 1

Related Questions