Python Selenium WebDriver unable to find element at all

Question

Selenium webdriver has been unable to find any elements on page using different methods: class_name, id, & xpath.

Here's my code:

from selenium import webdriver
##from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
import time
import random


chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")

driver = webdriver.Chrome(executable_path=r'C:\Users\acer\Downloads\chromedriver_win32\chromedriver.exe', chrome_options=chrome_options)
time.sleep(2)
driver.get('https://www.reddit.com/r/AskReddit/comments/fi04fh/what_are_some_spoilers_for_the_next_month_of_2020/')
time.sleep(2)
print(driver.title)
time.sleep(2)
element = driver.find_element_by_id("header")


print("done")

The title prints successfully but it fails on the line of driver.find_element_by_id("header"). In fact, I am trying to find the element whose class is "top-matter" (using find_by_class_name) but since this wasn't working, I tested it for other elements ("header") using respective methods ("xpath", "id") but nothing is working for me.

Can anyone provide some insight into the issue?

EDIT: Here's the error:

Traceback (most recent call last):
  File "C:/Python34/reddit_test.py", line 20, in <module>
    element = driver.find_element_by_id("header")
  File "C:\Python34\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 269, in find_element_by_id
    return self.find_element(by=By.ID, value=id_)
  File "C:\Python34\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 752, in find_element
    'value': value})['value']
  File "C:\Python34\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "C:\Python34\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"header"}
  (Session info: headless chrome=80.0.3987.132)
  (Driver info: chromedriver=2.41.578737 (49da6702b16031c40d63e5618de03a32ff6c197e),platform=Windows NT 6.1.7601 SP1 x86_64)

Here's proof that the element exists...

Answer 1

The issue was that I use the old version of reddit whereas the default which Selenium and Hamza were opening is the new version which doesn't contain the elements I was trying to find

Answer 2

in your url there is no header id

to ignore this exception

try this code:

from selenium import webdriver
##from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
import time
import random


chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")

driver = webdriver.Chrome(executable_path=r'C:\Users\acer\Downloads\chromedriver_win32\chromedriver.exe', chrome_options=chrome_options)
time.sleep(2)
driver.get('https://www.reddit.com/r/AskReddit/comments/fi04fh/what_are_some_spoilers_for_the_next_month_of_2020/')
time.sleep(2)
print(driver.title)
time.sleep(2)
try:
    element = driver.find_element_by_id("header")
except:
    print("The Header isd dose not exist!")
    exit()
print("done")

In Your Url The Header is dose not exist

You Can See By This Image!