CODEWITHSUNDEEP
pythonselenium-webdriverxpathcss-selectorswebdriverwait
930727fre
930727fre

Reputation: 41

How to crawl the title of the page?

I don't know how to crawl the title of the page,below is my code(it's simple),but I have no idea where is wrong, if you have any idea please let me know,thank you.

enter image description here

enter image description here

from selenium import webdriver
url="https://sukebei.nyaa.si/?s=seeders&o=desc&p=1"
driver_path = "C:\\webdriver\\chromedriver.exe"
option = webdriver.ChromeOptions()
driver = webdriver.Chrome(driver_path, options=option)
driver.implicitly_wait(10)
driver.get(url)
print(driver.find_element_by_xpath("/html/head/title").text)

Upvotes: 1

Views: 307

Answers (2)

undetected Selenium
undetected Selenium

Reputation: 193078

To crawl the title of the page you have to induce WebDriverWait for the visibility_of_element_located() for the <table> with torrent-list and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    driver.get('https://sukebei.nyaa.si/?s=seeders&o=desc&p=1')
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "table.torrent-list")))
print(driver.title)

  • Using XPATH:

    driver.get('https://sukebei.nyaa.si/?s=seeders&o=desc&p=1')
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//table[contains(@class, 'torrent-list')]")))
print(driver.title)

  • Console Output:

    Browse :: Sukebei

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Upvotes: 2

Fabix
Fabix

Reputation: 281

from selenium import webdriver
url="https://sukebei.nyaa.si/?s=seeders&o=desc&p=1"
driver_path = "C:\\webdriver\\chromedriver.exe"
option = webdriver.ChromeOptions()
driver = webdriver.Chrome(driver_path, options=option)
driver.implicitly_wait(10)
driver.get(url)
print(driver.title)

Upvotes: 1

Related Questions