Yu Na
Yu Na

Reputation: 122

Unable to click on element even after finding it- Python seleneium

I have been pained by a load more button for a while. I'm looking to create a loop where I click "load more" on the skills section of Linkedin pages. However, this button is just not consistently being clicked.

I was under the impression that the issue was that the element was not visible on the page. So, I have a segmented scroll, which continues moving down the page until the element is found. But what's baffling is that even though the page is now moving to the right place, the element is not being clicked. No error is being thrown.

I've tried nearly every version of the element location (xpath, class name, css selector, full xpath). Why would the button not be clicked, if it is visible on the page?

Relevant Code:

##log into Linkedin

linkedin_urls=['https://www.linkedin.com/in/julie-migliacci-revent/']

ChromeOptions = webdriver.ChromeOptions()
driver = webdriver.Chrome('C:\\Users\\Root\\Downloads\\chromedriver.exe')

driver.get('https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin')
driver.maximize_window()

WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.NAME, "session_key"))).send_keys("EMAIL")
WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.NAME, "session_password"))).send_keys("PASSWORD")
WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, "//button[@class='btn__primary--large from__button--floating']"))).click()


linkedin_urls=['https://www.linkedin.com/in/julie-migliacci-revent/', 'https://www.linkedin.com/in/kathleen-meyers-126a7931']


for linkedin_url in linkedin_urls:
   driver.get(linkedin_url)

   looking_for_element = True
   ldmore = ''

   while looking_for_element:
       elements = driver.find_elements_by_xpath('/html/body/div[7]/div[3]/div/div/div/div/div[2]/main/div[2]/div[6]/div/section/div[2]/button/span[1]')

       if len(elements) > 0:
          ldmore = elements[0]
          ldmore.click()
          looking_for_element = False
       else:
           global_copyright = driver.find_elements_by_css_selector('#globalfooter-copyright')

           if len(global_copyright) > 0:
               looking_for_element = False
           else:
               body = driver.find_element_by_css_selector('body')
               sleep(5)
               body.send_keys(Keys.PAGE_DOWN)

I've not seen a discussion on SO about element issues when the underlying solution is not visibility. The code is designed to stop once the element is located -- and is performing this correctly. But it just isn't clicking on the element. I'm not sure why this is.

Locations I've tried:

absolute xpath:
driver.find_element_by_xpath('/html/body/div[7]/div[3]/div/div/div/div/div[2]/main/div[2]/div[6]/div/section/div[2]/button/span[1]').click()

relative xpath: 
//span[contains(text(),'Show more')]

class name: 
pv-profile-section__card-action-bar pv-skills-section__additional-skills artdeco-container-card-action-bar artdeco-button artdeco-button--tertiary artdeco-button--3 artdeco-button--fluid" aria-controls="skill-categories-expanded

css: 
body.render-mode-BIGPIPE.nav-v2.theme.theme--classic.ember-application.boot-complete.icons-loaded:nth-child(2) div.application-outlet:nth-child(77) div.authentication-outlet:nth-child(3) div.extended div.body div.pv-profile-wrapper.pv-profile-wrapper--below-nav div.self-focused.ember-view div.pv-content.profile-view-grid.neptune-grid.two-column.ghost-animate-in main.core-rail div.profile-detail div.pv-deferred-area.ember-view:nth-child(6) div.pv-deferred-area__content section.pv-profile-section.pv-skill-categories-section.artdeco-container-card.ember-view div.ember-view > button.pv-profile-section__card-action-bar.pv-skills-section__additional-skills.artdeco-container-card-action-bar.artdeco-button.artdeco-button--tertiary.artdeco-button--3.artdeco-button--fluid

UPDATE: Tried the JS force, it clicked! But threw up the error: selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'click' of null

if len(elements) > 0:
    ldmore = elements[0]
    ldmorebtn = driver.find_element_by_xpath('/html/body/div[7]/div[3]/div/div/div/div/div[2]/main/div[2]/div[6]/div/section/div[2]/button/span[1]').click()
    #driver.execute_script("arguments[0].checked = true;", ldmore)
    driver.execute_script("arguments[0].click();", ldmore)

Upvotes: 1

Views: 193

Answers (1)

Yu Na
Yu Na

Reputation: 122

The suggestion by @Datanovice to use javascript to force a click worked like a charm. Initially, when I had tried to adapt the solution, I received the error selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'click' of null.

This error was because I had been using EC.element_to_be_clickable. Instead, when I paired the java method with EC.visibility_of_element_located, the click consistently worked.

The code:

ldmore = WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH,'xpath')))
driver.execute_script("arguments[0].click();", ldmore) 

Upvotes: 1

Related Questions