CODEWITHSUNDEEP
pythonseleniumacs
GeoG100
GeoG100

Reputation: 3

Selenium Python: Census ACS Data- unable to select Download button in window

I am attempting to scrape the Census website for ACS data. I have scripted the whole processes using Selenium except the very last click. I am using Python. I need to click a download button that is in a window that pops when the data is zipped and ready, but I can't seem to identify this button. It also seems that the button might change names based on when it was last run, for example, yui-gen2, yui-gen3, etc so I am thinking I might need to account for this someone. Although I normally only see yui-gen2.

Also, the tag seems to be in a "span" which might be adding to my difficulty honing in on the button I need to click.

Please help if you can shed any light on this for me.

code snippet: #Refine search results to get tables 

driver.find_element_by_id("prodautocomplete").send_keys("S0101")
time.sleep(2)
driver.find_element_by_id("prodsubmit").click()
driver.implicitly_wait(100)
time.sleep(2)
driver.find_element_by_id("check_all_btn_above").click()
driver.implicitly_wait(100)
time.sleep(2)
driver.find_element_by_id("dnld_btn_above").click()
driver.implicitly_wait(100)
driver.find_element_by_id("yui-gen0-button").click()
time.sleep(10)
driver.implicitly_wait(100)
driver.find_element_by_id("yui-gen2-button").click()

enter image description here

enter image description here

Upvotes: -1

Views: 97

Answers (3)

natn2323
natn2323

Reputation: 2061

Instead of using the element id, which as you pointed out varies, you can use XPath as Nogoseke mentioned or CSS Selector. Be careful to not make the XPath/selector too specific or reliant on changing values, in this case the element id. Rather than using the id in XPath, try expressing the XPath in terms of the DOM structure (tags):

//*/div/div/div/span/span/span/button[contains(text(),'Download')]

TIL you can validate your XPath by using the search function, rather than by running it in Selenium. I right-clicked the webpage, "inspect element", ctrl+f, and typed in the above XPath to validate that it is the Download button.

For posterity, if the above XPath is too specific, i.e. it is reliant on too many levels of the DOM structure, you can do something shorter, like

//*button[contains(text(),'Download')]

although, this may not be specific enough and may require an additional field, since there may be multiple buttons on the page with the 'Download' text.

Upvotes: 1

JeffC
JeffC

Reputation: 25531

Given the HTML you provided, you should be able to use

driver.find_element_by_id("yui-gen2-button")

I know you said you tried it but you didn't say if it works at all or what error message you are getting. If that never works, you likely have an IFRAME that you need to switch to.

If it works sometimes but not consistently due to changing ID, you can use something like

driver.find_element_by_xpath("//button[.='Download']")

Upvotes: 0

Nogoseke
Nogoseke

Reputation: 1009

On the code inspection view on Chrome you can right click on the item you want to find and copy the xpath. You can they find your element by xpath on Selenium.

Upvotes: 0

Related Questions