CODEWITHSUNDEEP
pythonhtmlseleniumselenium-webdriver
Jacob Neumann
Jacob Neumann

Reputation: 31

Python Selenium Webpage Scraping Selecting Drop Down and entering text in html form

I have a problem scraping data from the following site: https://arcc.sdcounty.ca.gov/Pages/Assessors-Roll-Tax.aspx.

I have to do these steps in order:

  1. Select a drop down option "Street Address'

  2. Enter a street address into a text field (ie 43 Hadar Dr)

  3. Click the 'Submit' button.

After clicking submit, I should be directed to a page that has the APN number for a given address.

The problem: I am able to do the above steps. However, when I select a drop down option and input address in the textbox, it fails as the textbox input address for some reason is cleared before clicking 'submit' ONLY when I have selected a drop down option.

I have tried using Selenium's Expected Conditions to trigger the input in the text box after a drop down option has been selected, but did nothing. I am looking for any help on identifying the why there is this problem as well as any advice on solutions.

Thanks.Much appreciated.

My code:

    driver = webdriver.Chrome()
    driver.get('https://arcc.sdcounty.ca.gov/Pages/Assessors-Roll-Tax.aspx')
    #Selects drop down option ('Street Address')
    mySelect =        Select(driver.find_element_by_id("ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25  ea12_ctl00_ddlSearch"))
    my=mySelect.select_by_value('0')  
    wait = WebDriverWait(driver,300)
    #Enter address in text box to left of drop down
   driver.find_element_by_id("ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12_ct    l00_txtSearch").send_keys("11493 hadar dr")
    #Click 'Submit' button to return API numbers associated with address
    driver.find_element_by_id("ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12_ctl00_btnSearch").click()
    driver.quit()

Upvotes: 1

Views: 1643

Answers (2)

A.Sherif
A.Sherif

Reputation: 144

Just changed a few things in your code to make it work.

mySelect = Select(driver.find_element_by_id("ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25  ea12_ctl00_ddlSearch"))

To find_element_by_name(...):

mySelect = Select(driver.find_element_by_name("ctl00$ctl43$g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12$ctl00$ddlSearch"))

And

my=mySelect.select_by_value('0')

To select_by_visible_text('...'):

my = mySelect.select_by_visible_text("Street Address")

And

driver.find_element_by_id("ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12_ct l00_txtSearch").send_keys("11493 hadar dr")

To find_element_by_xpath(...), since I usually get better results when finding elements by xpath.

driver.find_element_by_xpath('//*[@id="ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12_ctl00_txtSearch"]').send_keys("11493 hadar dr")

This is how it all looks like:

from selenium.webdriver.support.ui import WebDriverWait
from selenium import webdriver
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome()
driver.get('https://arcc.sdcounty.ca.gov/Pages/Assessors-Roll-Tax.aspx')

#Selects drop down option ('Street Address')
mySelect = Select(driver.find_element_by_name("ctl00$ctl43$g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12$ctl00$ddlSearch"))
my = mySelect.select_by_visible_text("Street Address")

wait = WebDriverWait(driver,300)

#Enter address in text box to left of drop down
driver.find_element_by_xpath('//*[@id="ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12_ctl00_txtSearch"]').send_keys("11493 hadar dr")

#Click 'Submit' button to return API numbers associated with address
driver.find_element_by_id("ctl00_ctl43_g_d30f33ca_a5a7_4f69_bb21_cd4abc25ea12_ctl00_btnSearch").click()

driver.quit()

Upvotes: 1

DVOPS
DVOPS

Reputation: 46

Not sure if this is your situation. But one thing that jumped out from your question is the text box input... Often, when filling in a website text box, even though the text is clearly visible, the text is not actually read by the text-box method until after the focus (cursor) is clicked or tabbed out and away from the text box.

Tabbing the text cursor out of the text entry box first, before 'clicking submit', will often solve this issue.

Upvotes: 0

Related Questions