Reputation: 3
So I am working on a selenium surf automation of this website ->
See attached image to understand the issue better
The problem im facing is, the code (see below) allows for two exact same (seemingly) ads to be true for the main if statement, which checks for the current element selected, having the proper attributes to comply with my codes purpose.
However the first ad on this page, becomes true for the main statement as it should, but then breaks after hitting the line right after it button_text = starting_element
. But the ad right after it, its able to traverse past that line and onto the rest of the nested loop etc.
This issue arises randomly (seemingly) amongst different ads throughout the page (of course the ones without any -call to action- button, being skipped, makes sense), but with no clear reason as to why.
while(True):
starting_element.send_keys(Keys.TAB)
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//body/div/div/div[@role='main']/div/div/div/div/div/div/div[4]/div[1]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]")))
starting_element = browser.switch_to.active_element
# check for a set of keywords when a CTA button is targeted, if matched then extract URL from source
if starting_element.get_attribute('role') == "button" and starting_element.get_attribute('aria-busy') == "false" and starting_element.get_attribute('tabindex') == '0':
button_text = starting_element.text
if button_text in meta_cta_buttons:
parent_element = starting_element.find_element(By.XPATH, "..")
while (True):
if parent_element.tag_name != 'a':
# moves up element ancestry chain
parent_element = parent_element.find_element(By.XPATH, "..")
else:
cta_url = parent_element.get_attribute('href')
# store links in a set
unique_store_urls.add(cta_url)
break
else:
continue
else:
continue
try:
# to look for the loading page data as part of infinite scroll
spinner_element = WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.XPATH, "//span[@role='progressbar']//*[name()='svg']")))
end_of_page_element = browser.find_element(By.XPATH, "//a[contains(text(),'Ad Library API')]")
if spinner_element:
print("Spinner exists")
time.sleep(3)
# if its at the footer, it means no data was loaded in time
elif starting_element == end_of_page_element:
break
except TimeoutException:
print("Spinner doesn't exist")
continue
I have printed out the result of the variables for the ad its breaking on, and that elements values are all correct. So im quite lost.
Upvotes: -2
Views: 63
Reputation: 3
Fixed my own problem.
Turns out the issue was with part of the code I didn't provide for more context in this question. It was the actual set
which was missing the keywords in lowercase to check for.
Thus the if statement was bypassing the ads quite simply.
Upvotes: 0