CODEWITHSUNDEEP
pythonseleniumselenium-webdriverweb-scraping
Anthony Spano
Anthony Spano

Reputation: 39

Selenium using WebDriverWait WebElement still doesn't print any text, recognizes string PYTHON

As title. Couldn't find a solution online. You'll see by the print statements that the program will get the text property from the WebElement object, but it is always an empty string even though I am using WebDriverWait.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

import time

PATH = "C:\\Users\\Anthony\\Desktop\\chromedriver.exe"

driver = webdriver.Chrome(PATH)

website = "https://www.magicspoiler.com"
driver.get(website)

el_name = '//div[@class="set-card-2 pad5"]'

try:
    main = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, el_name)))

    articles = main.find_elements(By.XPATH, el_name)
    
    for article in articles:
        print(type(article.text))
        print(article.text)

finally:
    driver.quit()

Upvotes: 0

Views: 66

Answers (1)

Anand Gautam
Anand Gautam

Reputation: 2101

I checked the HTML DOM and I don't see any text in the element you are looking for. They are simply images with anchor links and therefore you are getting blank responses. HTML DOM Snapshot

I have given an alternate try, to extract the links and they were successful. So, after refactoring your code, it looks like this:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

import time

PATH = "C:\\Users\\Anthony\\Desktop\\chromedriver.exe"

driver = webdriver.Chrome(PATH)
website = "https://www.magicspoiler.com"
driver.get(website)

el_name = '//div[@class="set-card-2 pad5"]/a'

try:
    main = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, el_name)))

    articles = main.find_elements(By.XPATH, el_name)

    for article in articles:
        # print(type(article.text))
        print(article.get_attribute("href"))

finally:
    pass
    driver.quit()

Here is the response I got: Response snapshot

So, please check if you want to get the links or really the text. If you want the text, then there is none that I see in DOM as I said. You may have to traverse through each of the link by clicking on it, and find if any you need fro the navigated page.

Upvotes: 1

Related Questions